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Abstract 


Conducting  the  right  amount  of  training  is  critical  and  a  scientific  basis  for  scheduling  refresher  or 
maintenance  training  is  required.  While  some  scientific  literature  exists  on  scheduling  of  individual 
training  on  procedural  tasks,  the  CF  operates  at  an  organisational  level  with  an  increasing  high  level  of 
technology  and  with  a  growing  emphasis  on  cognitive  tasks.  This  paper  reviews  published  and 
unpublished  data  and  theory  from  the  military  research  and  development  community  pertaining  to  the 
retention  of  collective  and  cognitive  skills  as  a  step  toward  fielding  a  validated  model  of  collective  and 
cognitive  skill  retention.  Electronic  sources  and  an  international  panel  of  scientists  were  consulted  to 
collect  available  data  for  review.  Review  of  the  selected  material  revealed  five  factors:  task  variables, 
structure  of  the  collective,  collective  membership,  individual  differences,  and  training.  Only  one 
developed  model  of  collective  skill  retention  was  found.  This  model  was  reviewed  in  conjunction 
with  other  models  related  to  retention  of  individual  skills.  None  are  broad  enough  in  scope  in  terms  of 
the  factors  covered  or  sufficiently  well  validated  to  serve  as  a  satisfactory  foundation  for  predicting 
retention  of  collective  performance.  A  plan  for  developing  a  model  of  collective  skill  retention  is 
proposed. 
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Resume 


II  est  crucial  d’offrir  le  bon  degre  de  formation,  et  il  faut  done  une  base  scientifique  pour 
l’etablissement  des  calendriers  des  cours  de  recyclage  ou  de  maintien.  II  existe  certains  ecrits 
scientifiques  sur  l’etablissement  de  calendriers  pour  la  formation  individuelle  portant  sur  des  taches 
procedurales,  mais  les  FC  fonctionnent  au  niveau  organisationnel,  en  faisant  appel  a  une  techno logie 
toujours  plus  perfectionnee  et  en  accordant  de  plus  en  plus  d’importance  aux  taches  cognitives.  Cet 
article  passe  en  revue  les  donnees  et  theories  publiees  et  non  publiees  provenant  du  milieu  de  la 
recherche  et  du  developpement  militaires  et  portant  sur  la  preservation  des  competences  collectives  et 
cognitives  en  tant  qu’etape  menant  a  la  proposition  d’un  modele  valide  de  maintien  des  competences 
collectives  et  cognitives.  On  a  consulte  des  sources  electroniques  et  un  groupe  international  de 
scientifiques  en  vue  d’identifier  les  donnees  disponibles  pour  la  recension.  L’analyse  de  la 
documentation  selectionnee  a  revele  1’ existence  de  cinq  facteurs  :  les  variables  des  taches,  la  structure 
du  collectif,  les  membres  du  collectif,  les  differences  individuelles  et  la  formation.  On  n’a  releve  qu’un 
modele  detaille  de  maintien  de  competences  collectives.  Ce  modele  a  ete  analyse  de  pair  avec  d’autres 
modeles  associes  au  maintien  de  competences  individuelles.  Aucun  n’a  une  portee  suffisamment  vaste 
en  termes  de  facteurs  pris  en  compte  ou  n’est  suffisamment  valide  pour  servir  de  fondement  a  la 
prevision  du  maintien  du  rendement  collectif.  Un  plan  d’elaboration  d’un  modele  de  maintien  des 
competences  collectives  est  propose. 
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Executive  Summary 


This  review  was  conducted  under  the  direction  of  Defence  Research  and  Development  Canada  - 
Toronto.  Within  the  ultimate  goal  of  fielding  a  validated  model  to  be  used  by  military  personnel  to 
schedule  training  of  collective  tasks,  the  purpose  of  this  project  was  to  review  publications  from  the 
military  research  and  development  community  concerning  retention  of  collective  and  cognitive  skills. 
Contributions  from  the  general  scientific  literature  were  also  considered. 

The  report  is  organized  around  sections  considering  training  in  the  military;  the  nature  of  collectives 
and  collective  tasks;  development  and  retention  of  collective  performance;  factors  affecting  retention 
of  collective  performance;  existing  models  of  collective  performance;  and  recommendations  for  a  way 
ahead. 

A  broad  range  of  factors  with  the  potential  to  affect  retention  of  performance  on  collective  tasks  was 
considered.  These  factors  were  categorized  as  those  affecting  the  collective  task  and  its  sub-tasks, 
collective  structure,  the  collective  membership,  individual  differences,  and  the  type  of  training 
provided  before  and  during  the  retention  interval  on  the  task  of  interest  and  others. 

A  brief  review  and  discussion  of  the  general  literature  on  team  work,  team  mental  models  and 
organizational  issues  suggests  that  retention  of  performance  on  individual  sub-tasks  will  be  strongly 
related  to  retention  of  overall  collective  performance  -  particularly  with  respect  to  co-ordination. 
Individual  sub-tasks  include  both  those  performed  in  isolation  by  collective  members  and  sub-tasks 
requiring  interaction  among  collective  members.  The  effect  of  these  individual  sub-tasks  on  overall 
collective  performance  will  depend  on  their  criticality  and  level  of  interdependence  with  respect  to  the 
overall  collective  task,  and  the  need  for  information  exchange  and  interaction  with  other  collective 
members.  Understanding  the  relationships  among  sub-tasks  within  particular  collective  tasks  for 
particular  collective  structures  is  a  pre-requisite  of  effective  study  of  retention  of  collective 
performance,  and  requires  selection  or  development  of  a  suitable  team  task  analysis  approach. 

Only  one  developed  model  of  collective  skill  retention  was  found  as  a  result  of  the  literature  search. 
This  model  was  reviewed  in  conjunction  with  other  models  related  to  retention  of  individual  skills. 

The  conclusion  was  drawn  that  while  these  models  form  a  good  conceptual  point  of  departure,  none 
are  broad  enough  in  scope  in  terms  of  the  factors  covered  or  sufficiently  well  validated  to  serve  as  a 
satisfactory  foundation  for  predicting  retention  of  collective  performance.  Particular  areas  of  omission 
included  transfer  from  other  training  during  the  retention  interval,  experience  of  team  members, 
internal  turbulence  and  external  turnover,  and  shared  team  mental  models. 

The  term  “collective”  was  rarely  found  in  the  literature  on  training  and  its  use  appears  to  add  little  to 
terms  such  as  team  and  organization,  already  in  common  use.  It  is  recommended  that  the  literature  on 
team  and  organizational  issues  be  more  thoroughly  reviewed  with  respect  to  the  issues  contained  in  this 
report.  In  particular,  the  literature  on  team  mental  models  should  be  considered  in  greater  detail  in 
relation  to  retention  issues,  including  retention  of  knowledge  contributing  to  team  performance. 

To  fulfil  the  global  goal  of  developing  an  effective  decision  support  tool  for  unit  planners  and 
modelling  retention  of  team  performance  under  conditions  typical  of  military  training,  a  longitudinal 
study  of  a  small  number  of  contrasting  collectives  and  collective  tasks  is  recommended.  This  should 
be  based  on  analysis  of  both  the  needs  of  unit  training  planners  and  the  team  tasks  involved  in  the 
collective  tasks  chosen. 
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Sommaire 

Cette  analyse  a  ete  effectuee  sous  la  direction  de  TIMED  (Institut  de  medecine  environnementale  pour 
la  defense).  Le  but  ultime  etant  de  proposer  un  modele  valide  pouvant  etre  utilise  par  le  personnel 
militaire  pour  planifier  la  formation  se  rapportant  aux  taches  collectives,  ce  projet  visait  a  passer  en 
revue  les  publications  du  domaine  de  la  recherche  et  du  developpement  militaires  portant  sur  le 
maintien  de  competences  collectives  et  cognitives.  On  a  egalement  pris  en  consideration  des  ouvrages 
scientifiques  generaux. 

Le  compte  rendu  est  divise  en  sections  portant  sur:  la  fonnation  dans  le  secteur  militaire;  la 
nature  des  collectifs  et  des  taches  collectives;  le  perfectionnement  et  le  maintien  d’un  rendement 
collectif;  les  facteurs  influant  sur  le  maintien  d’un  rendement  collectif;  les  modeles  existants  de 
rendement  collectif;  et  les  recommandations  quant  a  la  voie  a  suivre. 

On  a  pris  en  consideration  un  vaste  eventail  de  facteurs  susceptibles  d’influer  sur  le  maintien  du 
rendement  dans  Texecution  de  taches  collectives.  Ces  facteurs  ont  ete  repartis  en  categories:  ceux  qui 
influent  sur  les  taches  collectives  et  leurs  sous-taches,  la  structure  du  collectif,  les  membres  du  collectif, 
les  differences  individuelles  et  le  type  de  formation  offerte  avant  et  pendant  Tintervalle  de  maintien  a 
Tegard  de  la  tache  visee  et  d’autres  taches. 

Une  breve  etude  et  analyse  de  la  litterature  generate  sur  le  travail  d’equipe,  sur  les  modeles  mentaux 
d’ equipes  et  sur  les  questions  organisationnelles  donnent  a  penser  que  le  maintien  du  rendement  dans 
Texecution  de  sous-taches  individuelles  est  etroitement  lie  au  maintien  du  rendement  collectif  global  - 
particulierement  en  ce  qui  a  trait  a  la  coordination.  Les  sous-taches  individuelles  incluent  a  la  fois  celles 
qui  sont  executees  isolement  par  les  membres  d’un  collectif  et  cedes  qui  necessitent  une  interaction  entre 
les  membres  d’un  collectif.  L’effet  de  ces  sous-taches  individuelles  sur  le  rendement  collectif  global 
depend  de  leur  criticite  et  du  degre  d’interdependance  par  rapport  a  la  tache  collective  globale,  ainsi  que 
du  besoin  d’echange  d’information  et  d’interaction  avec  d’autres  membres  du  collectif.  II  faut  au 
prealable  comprendre  les  liens  entre  les  sous-taches  de  taches  collectives  particulieres  dans  des  structures 
collectives  particulieres  pour  pouvoir  bien  etudier  le  maintien  du  rendement  collectif,  ce  qui  necessite  la 
selection  ou  la  mise  au  point  d’une  methode  convenable  d’analyse  des  taches  d’equipe. 

La  revue  de  la  litterature  n’a  permis  de  decouvrir  qu’un  modele  detaille  de  maintien  des  competences 
collectives.  Ce  modele  a  ete  examine  parallelement  a  d’autres  modeles  de  maintien  des  competences 
individuelles.  On  en  est  arrive  a  la  conclusion  que,  meme  si  ces  modeles  constituent  un  point  de  depart 
conceptuel  interessant,  aucun  n’a  une  portee  suffisamment  vaste  en  termes  de  facteurs  pris  en  compte 
ou  n’est  suffisamment  bien  valide  pour  servir  de  fondement  a  la  prevision  du  maintien  du  rendement 
collectif.  Les  lacunes  ont  ete  observees  particulierement  dans  les  domaines  suivants:  les  echanges 
entre  activites  de  formation  au  cours  de  Tintervalle  de  maintien,  T  experience  des  membres  des 
equipes,  les  perturbations  internes  et  le  roulement  exteme,  ainsi  que  les  modeles  mentaux  communs 
des  equipes. 

Le  terme  «collectif/collective»  a  rarement  ete  releve  dans  la  litterature  sur  la  formation  et  son  emploi 
ne  semble  pas  ajouter  beaucoup  a  des  termes  comme  «equipe»  et  «organisation»  deja  couramment 
utilises.  II  est  recommande  d’examiner  plus  a  fond  la  litterature  sur  les  equipes  et  les  organisations, 
sous  Tangle  des  points  souleves  dans  ce  compte  rendu.  II  faudrait  surtout  etudier  plus  attentivement  la 
litterature  sur  les  modeles  mentaux  des  equipes  par  rapport  a  Taspect  du  maintien,  et  notamment  du 
maintien  des  connaissances  contribuant  au  rendement  des  equipes. 

Pour  atteindre  Tobjectif  global,  qui  est  d’elaborer  un  bon  outil  d’aide  a  la  decision  a  Tintention  des 
planificateurs  d’unites  et  de  modeliser  le  maintien  du  rendement  d’equipe  dans  des  conditions 
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caracteristiques  de  la  formation  militaire,  il  est  recommande  d’effectuer  une  etude  longitudinale  sur  un 
petit  nombre  de  collectifs  et  de  taches  collectives  divergents.  II  faudrait  a  cette  fin  se  fonder  sur 
l’analyse  des  besoins  des  planificateurs  de  la  formation  des  unites  ainsi  que  des  taches  d’equipe 
composant  les  taches  collectives  choisies. 
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Introduction 


This  project  was  conducted  under  the  direction  of  Defence  Research  and  Development  Canada  - 
Toronto  by  Human systems®  Incorporated  under  Standing  Offer  contract  W771 1-0-7694,  let  on  1st 
January  2002  and  completed  on  March  3 1st  2002. 

The  Canadian  Forces  (CF)  has  large  and  ongoing  requirement  to  train.  Conducting  the  right  amount 
of  training  is  critical  and  a  scientific  basis  for  scheduling  refresher  or  maintenance  training  is  required. 
While  some  scientific  literature  exists  on  scheduling  of  such  training  at  the  individual  level  on 
procedural  tasks,  the  CF  operates  at  an  organizational  level  with  an  increasing  high  level  of  technology 
and  with  a  growing  emphasis  on  cognitive  tasks.  Hence,  a  scientific  basis  for  conceptualising  and 
predicting  the  retention  of  performance  on  collective  tasks  is  required. 

This  issue  has  also  been  recognized  by  Canada’s  allies,  and  some  theory  development  and  data 
collection  has  occurred,  but  a  well  developed  approach  has  not  emerged. 

1.1  Purpose 

The  ultimate  goal  is  to  field  a  validated  model  to  be  used  by  military  personnel  to  schedule  the  training 
of  collective  tasks.  The  purpose  of  this  project  was  to  review  published  and  unpublished  data  and 
theory  from  the  military  research  and  development  community  pertaining  to  the  retention  of  collective 
and  cognitive  skills  together  with  contributions  from  the  general  scientific  literature. 

1 .2  Organization  of  the  Report 

The  report  is  organized  in  the  following  sections,  starting  with  the  present  one. 

•  Statement  of  Work,  deliverables  and  approach. 

•  Brief  background  to  training  in  military. 

•  Collectives  and  collective  tasks 

•  Collective  skill  development  and  retention 

•  Factors  affecting  collective  skill  retention 

•  Existing  models  of  collective  skill  retention 

•  A  way  ahead 

1.3  Work  Items 

The  following  work  items  were  conducted. 

•  Search  and  review  available  literature,  incorporating  available  TTCP  reports. 

•  Develop  a  categorisation  framework  and  analyse  models  found  in  the  literature. 

•  Develop  a  prediction  approach  for  selected  factors  related  to  collective  and  cognitive  skill 
retention. 

•  Prepare  a  report  and  make  a  presentation. 
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1.4  Deliverables 

•  Fifteen  hard  copies  of  the  work  prepared  according  to  TTCP  publication  format. 

•  One  copy  of  an  MS  Poweipoint  presentation  summarising  the  work. 

•  One  copy  of  the  work  in  MS  Word  on  CD-ROM. 

1.5  Approach 

Our  main  focus  has  been  on  research  in  military  domain,  and  on  the  collective  skills  and  tasks  relevant 
to  the  military  context.  We  have  assumed  that  the  eventual  system  for  predicting  collective  skill 
retention  must  be  workable  within  the  established  systems  of  military  training. 

1.6  Method 

1.6.1  Develop  Framework 

The  first  step  of  this  literature  review  was  to  develop  a  framework  for  organising  the  search  and 
interpreting  the  results.  A  related  goal  was  to  characterize  the  nature  of  collective  training  (and 
collective  tasks)  within  the  military  domain. 

Factors  with  potential  relevance  to  collective  and  cognitive  skill  retention  were  grouped  and  discussed 
in  the  report  as  follows. 

•  Nature  of  the  collective  and  cognitive  task 

•  Nature  of  the  collective 

•  Individual  differences  within  the  collective 

•  Collective  training  factors 

The  first  set  of  factors  relates  to  the  unique  features  of  collective  tasks.  Several  features  of  collective 
tasks  likely  to  impact  on  collective  skill  retention  were  explored,  including  complexity  of  the  task,  and 
the  extent  to  which  a  task  requires  extensive  co-ordination  for  successful  completion.  Most  research 
in  skill  retention  has  been  devoted  to  perceptual-motor  skills,  and  the  need  for  research  in  retention  of 
cognitive  tasks  is  discussed. 

The  second  set  of  factors  concern  the  nature  of  the  collective  such  the  size  of  the  collective, 
organizational  structure,  and  the  social  environment  within  it. 

The  third  set  of  factors  relates  to  individual  differences  such  as  aptitude  and  expertise  evident  within 
the  collective. 

The  fourth  set  of  factors  considers  training  related  issues  such  as  training  strategies,  knowledge  of 
results,  characteristics  of  the  retention  interval,  type  and  scheduling  of  practice  in  key  collective  tasks. 

1.6.2  Consult  TTCP  members 

Contact  with  members  of  the  TTCP  committee  was  limited  by  the  time  available  and,  in  general,  the 
responses  confirmed  the  outcome  of  the  literature  search  i.e.  that  there  is  little  relevant  work  available. 


Page  2 


Collective  and  Cognitive  Skill  Retention 


Y\umdx\systems  Incorporated 


1.6.3  Search  Literature 


Relevant  databases  providing  access  to  information  about  collective  and  cognitive  skill  retention  in  the 
military  domain,  as  well  as  in  the  broader  scientific  domain  were  searched  including  psychology, 
human  factors,  military,  business  and  other  related  domains. 

A  comprehensive  set  of  keywords  was  selected  to  focus  and  guide  the  search,  based  on  experience  in 
conducting  a  previous  skill  retention  literature  review  and  on  input  from  TTCP  committee  members. 

These  keywords  were  then  applied  to  the  relevant  databases  and  roughly  150  potentially  relevant 
abstracts  and  titles  were  identified.  These  abstracts  and  titles  were  filtered  in  terms  of  their  scope,  level 
of  detail,  and  relevance  to  the  problem  of  collective  skill  retention.  The  literature  that  we  retrieved 
during  the  search  came  from  many  different  areas,  including  the  industrial/organizational  literature  and 
the  team  performance  literature.  Although  there  were  many  articles  that  touched  on  collective  skill 
retention,  to  keep  within  the  scope  of  the  review,  a  much  smaller  set  of  articles  were  deemed  to  directly 
address  this  issue.  Of  these  articles,  20  articles  were  selected  for  detailed  review  and  these  became  the 
basis  of  this  report. 

1.6.4  Review  Selected  Articles 

Each  of  the  twenty  articles  selected  were  reviewed  with  respect  to: 

•  Factors  and  issues  relevant  to  collective  and  cognitive  skill  retention 

•  Existing  theories  and  models  relevant  to  collective  and  cognitive  skill  retention 

•  Developing  an  approach  to  predicting  collective  and  cognitive  skill  retention 

1.6.5  Generate  Survey  of  Literature  and  Lessons  Learned 

We  evaluated  the  models  in  the  articles  found  with  respect  to  the  scientific  validity  and  applicability  to 
the  military  TTCP  domain.  More  specifically,  our  analysis: 

•  Reviewed  existing  models  with  respect  to  the  categorization  framework  to  understand 
their  strengths  and  weaknesses. 

•  Suggested  alterations  to  the  framework,  and  identified  the  need  for  more  data  to  provide 
direction  for  research  and  theory  needed  to  further  models  of  collective  skill  retention. 

•  Generated  recommendations  for  elaborating  existing  theories  of  collective  skill  retention  and 
new  approaches  to  studying  collective  skill  retention. 
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1.8  Acronyms 


ARI 

Army  Research  Institute 

NTIS 

National  Technical  Information  Service 

CF 

Canadian  Forces 

R&D 

Research  and  Development 

CISTI 

Canadian  Institute  for  Scientific  and 
Technical  Information 

SA 

Scientific  Authority 

CTEP 

Collective  Task  Effect  Prediction 

SME 

Subject  Matter  Expert 

DCIEM 

Defence  and  Civil  Institute  of 
Environmental  Medicine 

TTCP 

Technical  Cooperation  Program 

JRTC 

Joint  Readiness  Training  Center 

UDA 

User's  Decision  Aid 

MOS 

Military  Occupational  Specialty 

UTEP 

Unit  Type  Effect  Prediction 

Table  1:  List  of  Acronyms 
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Background 


This  project  was  conducted  within  the  context  of  a  Canadian  model  for  military  training  for  combat 
arms  skills,  but  the  general  intent  is  for  this  work  to  be  applicable  to  any  NATO  military  training: 
army,  navy  or  airforce. 

Military  training  covers  a  wide  variety  of  skills  for  both  individual  tasks  involving  different 
combinations  of  perceptual-motor  and  cognitive  skills  and  team  or  collective  tasks  that  require  several 
individuals  or  teams,  often  with  different  skills  and  performing  complementary  tasks  requiring  split 
second  interaction.  Co-ordination  and  information  exchange  are  important  aspects  of  most  collective 
tasks. 

2.1.  Military  Approach  to  Training 

Skill  retention,  and  any  refresher  training  to  offset  skill  decay,  needs  to  be  set  in  the  general  context  of 
military  training.  In  outline,  a  military  career  progresses  from  training  on  individual  skills  and  tasks  to 
team  (or  small  collective)  tasks  to  large  collective  tasks.  Recruit  training  in  basic  individual  military 
skills  and  some  basic  special  to  arm  skills  will  be  followed  by  unit  training  as  part  of  a  team  such  as  a 
vehicle  crew,  or  infantry  section.  Generally  speaking,  collective  training  starts  within  a  single  purpose 
organization  (such  as  an  infantry  company  or  armoured  squadron).  Finally,  comes  training  as  a  multi¬ 
purpose  group  (e.g.  as  a  combat  team  comprising  artillery,  infantry,  armour,  engineers,  logistic  units). 
A  similar,  progressive,  approach  to  training  characterizes  training  in  other  service  organizations,  and 
joint  service  training. 

Annual  unit  training  cycles  also  follow  a  progressive  pattern,  often  starting  after  an  annual  period  of 
postings  and  vacations  (often  in  the  summer)  and  specialist  or  trade  training  away  from  the  unit.  The 
annual  cycle  commonly  begins  with  individual  skills  and  progresses  through  team  or  sub-unit  training 
up  to  formation  training.  Mission  training  may  require  up  to  six  months  of  focussed  training  prior  to 
deployment.  A  between  year  rotational  cycle  may  also  be  established  with  one  year  on  a  mission  or  as 
part  of  a  rapid  deployment  force,  one  year  reconstituting  after  a  mission,  and  one  year  spent  in 
collective  training  and  mission  preparation.  This  general  approach  is  illustrated  below  in  Figure  l.1 


1  The  relationship  between  “teams”  and  “collectives”  is  discussed  in  more  detail  later  in  the  report. 
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Period  in  military  career 


Course 

Course 

Periodic 

Prog  ress 

Readiness 

Equation 

Equation 

Evaluation 

Evaluation 

Evaluation 

Generic 

Mission 

MOPS 

MOPS 

Figure  1:  General  training  framework  (mop:  Measure  of  Performance) 


2.2.  Constraints  Affecting  Training 

The  quality  and  amount  of  training  will  be  limited  by  a  number  of  budgetary  and  practical  constraints. 
These  constraints  include  equipment  availability  and  maintenance,  ammunition,  accommodation  and 
ground  /  air  transportation,  personnel  and  logistic  costs.  In  peacetime  it  may  be  difficult  and/or 
expensive  to  achieve  appropriate  levels  of  realism  to  represent  the  stress  of  battle  by  using  live  or 
simulated  ammunition.  Additional  training  of  supplementary  personnel,  possibly  from  reserve  units, 
may  be  required  to  bring  peacetime  organizations  to  war-time  strength  and  capability.  There  is 
increasing  use  of  training  simulators  for  individual  and  collective  skills  -  sometimes  linking  widely 
distributed  simulators  in  different  locations.  Environmental  regulations  may  also  limit  the  type  of 
training  that  be  conducted  in  certain  areas. 


Figure  2:  Armoured  Vehicle  Crew  Training 
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2.3.  Reference  Examples 

As  reference  points  during  the  review  and  discussion  of  the  issues  of  interest  in  this  report,  we  have 
kept  in  mind  two  examples  of  hypothetical  teams  or  collectives.  One  example  is  a  three  person  tank 
crew  comprising  a  gunner,  driver,  and  a  vehicle  commander.  This  example  allows  consideration  of 
basic  individual  skills  of  each  of  the  three  crew  positions,  as  well  as  the  interaction  of  these  skills. 
Other  skills  related  to  the  performance  of  a  larger  collective  (i.e.  several  vehicle  crews  working 
together)  can  also  be  considered.  Selection  of  a  tank  crew  as  an  example  also  allows  consideration  of 
the  effect  of  various  permutations  and  combinations  of  organizational  level. 

The  other  example  is  a  hypothetical  operations  room  in  a  naval  frigate.  The  operation  of  a  frigate  is 
controlled  by  an  operations  room  team  of  some  twenty  personnel.  This  team  receives  radar, 
electronic,  and  acoustic  data  from  various  sensors,  as  well  as  visual  reports,  intelligence  information 
and  message  traffic.  This  activity  is  directed  by  an  operations  room  officer  working  through  the 
directors  of  two  warfare  teams:  one  for  above  surface  operations  and  one  for  below  surface  operations. 
The  command  team  comprises  the  ship’s  captain,  the  operations  room  officer  and  the  two  warfare 
directors.  This  example  permits  consideration  of  collective  issues  requiring  the  co-ordination  of 
diverse  tasks  and  skills.  Such  teams  operate  at  the  perceptual  and  cognitive  level  for  the  purpose  of 
co-ordinating  subordinate  levels  within  the  ship,  but  also  at  a  broader  level  working  to  co-ordinate  the 
activity  of  the  ship  in  relation  to  other  units  of  a  larger  collective  or  task  group. 
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3.  Collectives  &  Collective  Skills  and  Tasks 


Our  first  observation  is  that  the  term  “ collective ”  in  relation  to  skills  and  tasks  does  not  appear  to  be  in 
general  use  in  the  scientific  literature.  A  search  using  “collective”  as  a  key  word  revealed  only  two 
relevant  articles.  Where  it  is  used,  its  application  seems  largely  irrelevant  to  the  focus  of  this  review. 
In  social  psychology,  for  instance,  the  term  refers  to  the  behaviour  such  as  football  crowds,  riots,  panic 
during  escapes,  stock  market  runs,  and  so  (Brown,  1965).  The  term(s)  of  choice  in  the  literature  are 
team,  crew,  group,  organization  but  even  these  are  seldom  used  in  relation  to  skill  retention.  The  term 
“ collective ”  as  used  to  describe  the  focus  of  this  review  appears  to  be  uniquely  military,  with  no 
authoritative  definition  found.  Roth  (1992)  for  example,  in  the  only  study  found  on  collective  training 
as  such,  conducted  for  the  U.S.  Army,  attempts  no  definitional  framework.  This  begs  the  question: 
does  the  use  of  the  term  add  any  value  over  the  use  of  existing  terms,  or  does  it  just  add  confusion? 

The  purpose  of  this  section,  therefore,  is  to  discuss  the  nature  of  “collectives”  and  “collective”  skills 
and  tasks,  in  relation  to  other  terms  used  in  the  literature. 

3.1  Defining  Collectives 

The  diagram  below  illustrates  a  simple  hypothetical  “collective”  comprising  three  “Units”,  each  unit 
comprising  three  “teams”  or  crews,  and  each  team,  three  individuals.  Among  the  group  of  three,  there 
is  assumed  to  one  position  that  has  a  leadership  or  co-ordination  role.  Thus,  each  crew  has  a  leader  or 
commander,  and  those  commanders  co-ordinate  between  them  the  activities  of  the  unit  as  a  whole.  In 
turn,  each  unit  has  one  commander  who  co-ordinates  with  the  commanders  of  other  units  to  control  the 
activities  of  the  collective  as  a  whole.  This  analysis  can  be  pursued  up  through  successive 
organizational  levels  ad  infinitum  but,  at  some  point,  a  specialized  command  or  co-ordination  role  may 
emerge.  This  is  not  represented  in  the  diagram  but,  in  reality,  would  occur  and  be  represented  by  a 
specialized  “commander”  with,  in  larger  collectives,  a  team  of  specialized  “staff’  to  support  to  the 
command  and  co-ordination  role. 

Although  three  is  chosen  for  the  number  of  members  at  each  level  in  the  collective,  this  is  solely  for 
simplicity,  and  the  range  of  numbers  that  actually  or  best  occur  at  any  level  within  the  collective 
depends  on  the  allocation  of  functions  with  respect  the  tasks  it  has  to  perform,  and  the  equipment  with 
which  it  is  provided.  Moreover,  as  numbers  at  any  level  of  a  collective  grow,  it  seems  likely  that  the 
total  number  will  be  become  decomposed  into  sub-groups  that  best  fit  the  functional  goals  in  question, 
or  the  scope  of  the  ability  to  co-ordinate.  The  way  in  which  that  happens  remains,  and  how  it  might  be 
idealized,  remains  an  open  question. 

In  the  illustration,  it  is  assumed  that  each  “unit”  can  have  a  different  function  (e.g.  within  an  army 
combat  team  an  armoured  squadron,  infantry  company  and  engineer  troop  or  within  a  ship, 
departments  such  as  operations,  combat  systems,  or  propulsion  systems).  To  fulfil  the  unit  function, 
the  composite  crews  may  be  either  more  or  less  identical  or  more  or  less  specialized. 

Consideration  of  this  hypothetical  collective  leads  to  the  observation  that  collectives  may  operate  at 
multiple  levels,  more  or  less  simultaneously  i.e.  within  each  team,  among  the  crew  leaders  with  each 
unit,  and  so  on.  Formal  or  informal  co-ordination  may  also  take  place  among  individuals  with 
common  specialities  spread  among  different  teams  or  units.  For  instance,  the  underwater  warfare 
specialist  teams  in  different  ships  may  co-ordinate  technical  information  laterally  and  operational 
information  vertically  within  the  command  structure. 
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The  term  “ collective ”,  as  defined  in  the  Canadian  Dictionary  of  the  English  Language,  denotes  “a 
number  of  people  acting  as  a  group”  (ITP  Nelson,  1997).  However,  this  simple  definition  is 
essentially  passive  and  misses  the  rich  range  of  potential  organizational  options  and  the  degree  of 
interdependence  and  co-ordination  required,  depending  on  the  task(s)  of  the  collective  (see  next 
section). 

Thus,  the  term  collective  can  be  understood  at  varying  levels  of  analysis.  At  the  simplest  level, 
individuals  are  the  “building  blocks”.  At  the  next  level,  “team”  or  “crew”  is  a  commonly  used  term 
with  the  core  features  of  a  team  often  defined  as  two  or  more  individuals  who  work  together  in  order 
to  achieve  a  specific  goal  (Dieterly,  1988).  At  a  higher  level  yet,  multiple  teams  can  combine  to  form 
a  larger  collective. 

Thus,  it  is  difficult  to  say  what  distinguishes  a  team  from  a  collective,  or  a  collective  from  an 
organization.  The  inherent  aspects  of  teams  are  that  their  members  interact  frequently  to  conduct 
mutually  interdependent  tasks  in  pursuit  of  a  common  goal.  The  inherent  aspect  of  an  organization 
seems  to  be  that  the  unit  of  analysis  shifts  from  the  individual  to  the  team:  a  team  of  teams.  This 
involves  a  division  of  labour  and  some  distribution  or  hierarchy  of  authority  in  pursuit  of  a  common  goal 
or  goals  (Hendrick,  1977).  It  is  not  readily  apparent  what  the  concept  of  a  collective  adds  to  this. 

One  may  regard  a  collective  (or  organization  or  team)  as  any  set  of  two  or  more  individuals  or  teams 
that  work  together  for  the  specific  purpose  of  accomplishing  a  task.  A  team  might  indicate  a  smaller 
collective,  and  a  unit  or  sub-unit  a  larger  collective,  but  that  there  is  really  no  upper  limit  to  the  size  or 
complexity  of  a  collective.  However,  this  poses  a  practical  problem  in  relation  to  measurement  of  skill 
decay.  If  one  chooses  outcome  measures  collective  by  collective,  there  will  be  virtually  no  limit  in 
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measures  of  performance.  There  needs  to  be  a  measurement  set  for  each  type  and  level  of  collective. 
This  suggests  that  it  would  be  preferable,  from  a  practical  point  of  view,  to  choose  process  measures 
related  to  collective  skills  held  in  common  across  a  range  of  collective  structures  and  tasks. 

The  structure  of  collectives  may  be  differentiated  along  a  number  of  dimensions  -  size,  permanence 
(i.e.  duration  of  membership  of  a  collective  -  as  an  individual  or  as  a  team),  interdependence  of 
component  parts  (individuals,  teams  or  units),  level  of  specialisation,  and  the  co-ordination  required. 
Less  tangible  but  nevertheless  important  factors  may  also  arise  such  as  the  trust  and  motivation  within 
and  between  groups.  Many  of  these  dimensions  have  the  potential  to  affect  skill  retention  (or  related 
performance). 

Because  research  relevant  to  collective  skill  retention  has  focused  mainly  at  the  small  team  level 
(Tumage  et  al.,  1990),  this  review  has  tended  to  concentrate  on  smaller,  more  enduring  and  coherent 
collectives.  In  part,  also,  this  is  because  the  defining  characteristic  of  collectives  appears  to  be  the 
need  for  co-ordination  and  information  exchange  among  mutually  dependent  different  elements  to 
achieve  a  specific  goal.  In  turn,  largely  cognitive  functions  of  goal  setting,  planning  and  co-ordination 
may  themselves  be  conducted  by  specialized  teams  in  larger  collectives,  and  “virtual  teams” 
comprising  team  leaders  in  smaller  collectives.  As  our  review  will  explore,  different  types  of 
collectives  may,  by  virtue  of  their  structure,  size  etc.  may  be  subject  to  unique  factors  that  affect  skill 
retention. 

3.2  Defining  Collective  Skills  and  Tasks 

Turning  from  the  structure  of  the  collective  to  the  tasks  they  perform,  a  point  of  departure  is  the 
relationship  between  the  constructs  of  “task”  and  “ skill ”,  both  of  which  are  used  frequently  in  the 
literature  on  training,  acquisition  and  retention. 

Commonly  a  “task”  refers  to  a  piece  of  work  to  be  done,  and  is  typically  depicted  as  a  specific  sequence 
of  steps  to  be  achieved.  Since  most  collectives  may  be  called  on  to  perform  more  than  one  task,  there 
are  far  more  tasks  than  collectives.  This  poses  a  practical  problem  for  measuring  collective 
performance  at  the  task  level:  there  are  very  many  and  they  are  very  varied. 

For  each  collective  task,  there  will  be  a  number  of  sub-tasks  that  have  to  be  performed  by  individuals 
or  sub-units  within  the  collective.  Some  of  these  sub-tasks  may  require  information  exchange  among 
collective  members. 
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Figure  4:  A  hypothetical  collective  task  and  sub-tasks 

The  interdependence  of  these  tasks  in  terms  of  speed,  accuracy,  and  timeliness  leads  to  the  idea  of  sub¬ 
task  “ criticality ”  as  important  and  relevant  to  the  analysis  of  collective  work.  Some  sub-tasks  will 
need  to  be  performed  in  sequence,  to  a  certain  level  of  accuracy,  within  a  certain  time  window.  If 
these  sub-tasks  are  delayed  or  performed  with  sub-standard  results,  then  the  performance  of  the 
collective  task  as  a  whole  will  be  affected  to  a  greater  or  lesser  degree.  Other  sub-tasks  will  be  less 
critical.  Thus  “criticality”  can  stretch  from  having  severe  or  fatal  effects  on  collective  task 
performance  to  minor  or  trivial  effects.  Retention  for  non-critical  tasks  can  be  expected  to  have  less  of 
an  effect  on  overall  collective  performance.  This  observation  suggests,  in  turn,  that  some  form  of  team 
or  collective  task  analysis  would  be  beneficial  in  predicting  the  impact  of  skill  retention  on  collective 
tasks.  This  is  the  view  of  Dieterley  (1988)  who  also  notes  the  absence  of  such  an  analysis  method  in 
the  literature  and  the  need  to  establish  measures  such  as  a  sub-task  interaction  ratio  as  an  index  of 
interdependence  among  critical  sub-tasks  within  overall  collective  tasks.  Dieterley  also  raises  the 
issue  of  function  allocation  within  teams,  pointing  out  that  function  allocation  among  team  members 
will  determine  the  interaction  ratio  -  and  that  function  allocation  is  a  matter  of  initial  system  design, 
and  its  subsequent  adaptation  to  different  circumstances. 

A  “ skill  ”,  on  the  other  hand,  refers  to  the  generic  underlying  ability  or  abilities  needed  to  perform  an 
individual  task  or  range  of  tasks  (such  as  keyboard  skills  or  marksmanship),  or,  alternatively,  to  the 
manner  in  which  the  task  is  performed  (well  or  poorly).  For  teams  and,  by  extension,  collectives, 
skills  can  be  separated  into  individual  skills  required  for  individual  sub-tasks  in  the  team  and 
collective  skills  for  tasks  required  for  performance  as  unit  members  -  such  as  information  exchange, 
planning  and  co-ordination  (Roth,  1992).  Morgan  et  al.  (1986;  cited  in  Swezey  and  Llaneras,  1997) 
makes  a  further  distinction  between  task  work  skills  (individual  ability  to  perform  tasks)  and  team 
work  skills  (the  group’s  ability  to  perform  together).  This  implies  measurement  of  skills  at  both  the 
individual  and  the  team  level. 


Page  12 


Collective  and  Cognitive  Skill  Retention 


Y\ummsystems  Incorporated 


Others  (e.g.  Rasmussen,  1990;  Vicente,  1999)  separate  skill,  rule  and  knowledge  based  behaviours. 
Skill  based  behaviours  are  simple  stimulus-response  sequences  that  can  be  learned  by  frequent 
rehearsal  and  become  highly  automated.  Rule  based  behaviours  represents  routine  stimulus  response 
action  sequences  that  have  became  automatic  through  long  practice;  rule  based  behaviours  represent 
conscious  choices  based  on  unambiguous  procedural  decision  rules  requiring  the  ability  to  distinguish 
clearly  between  situations.  Knowledge  based  behaviours  require  reasoning  based  on  some  symbolic 
mental  representation  of  the  problem  space  in  question,  in  the  absence  of  familiar  cues  and  clear 
precedents.  These  and  other  somewhat  similar  formulations  can  be  related  to  the  stages  of  learning 
(e.g.  Anderson,  1982;  Gagne,  1965)  whereby  the  stage  of  acquisition  progresses  from,  for  example,  a 
cognitive  associative  stage  using  declarative  knowledge  to  develop  rules  to  govern  behaviour,  and 
eventually  reaching  an  autonomous  stage  whereby  procedures  become  automatic.  Retention  of 
individual  skills  is  thought  by  some  to  depend  on  the  level  of  learning  reached.  Plateaux  in 
performance  observed  during  training  have  sometimes  been  thought  to  relate  to  different  acquisition 
stages  (Swezey  and  Llaneras,  1997).  If  this  is  so,  one  might  speculate  about  the  existence  of  plateaux 
in  decay,  possibly  relating  to  qualitatively  different  stages  of  decay. 

Within  military  collectives,  individual  sub  tasks  may  be  performed  largely  independently  by  members 
of  the  collective.  Many  more,  however,  require  co-ordination  among  interdependent  tasks  performed 
by  other  team  members  to  complete  the  task  effectively. 

However,  not  every  component  of  a  collective  task  is  necessarily  shared,  and  individual  skills  as  well 
as  team  or  collective  skills  can  be  critical  in  performing  some  collective  tasks.  Some  of  the  tasks 
performed  by  collectives  could  be  seen  as  several  different  individual  tasks  with  very  little  overlap,  all 
of  which  must  completed  before  completion  of  the  overall  collective  task.  At  issue  is  the  extent  to 
which  performance  of  any  given  collective  overall  task  is  influenced  by  the  completion  of  component 
team  or  individual  sub-tasks,  and  therefore  affected  by  decay  in  either  team  or  individual  skills.  The 
implication  is  that  understanding  retention  at  the  individual  level  is  likely  an  important  part  of 
understanding  collective  retention  but  cannot,  on  its  own,  provide  an  accurate  account  of  skill 
retention  within  collective  contexts. 

3.3  Collective  Cognitive  Skills  and  Tasks 

For  individual  skills,  some  (e.g.  Swezey  and  Llaneras,  1997;  Sabol  and  Wisher,  2001)  use  the  term 
“cognitive”  to  refer  to  the  decision  making  phase  in  the  sequence  of  performing  any  task  performance: 
i.e.  first  knowledge  retrieval  (including  memory  for  procedures),  then  decision  making  (as  in  fault 
diagnosis  for  equipment,  and  tactical  planning),  and  finally  execution  (primarily  psycho-motor  or 
perceptual  activities,  such  as  tracking  a  target). 

Recent  work  (e.g.  Salas  and  Cannon-Bowers,  2001)  models  the  cognitive  aspects  of  team  work  (as 
opposed  to  individual  task  work)  in  terms  of  the  degree  to  which  team  members  shared  a  common 
understanding  and  knowledge  of  the  team  task,  the  capabilities  of  team  members  and  the  context  in 
which  the  work  is  to  be  performed.  Swezey  et  al.  (1994)  suggest  that  there  are  four  re-requisites  for 
effective  team  performance:  information  exchange,  effective  co-ordination,  periodic  adjustment  to 
changes  in  task  demands,  and  an  agreed  organization  within  the  group.  Matthews  and  Webb  (1993) 
concluded  that  the  three  key  processes  to  assess  for  command  and  control  were  effective 
communication  or  information  exchange,  situation  awareness  and  decision  making. 

Knowledge  acquisition  (for  instance  for  shared  team  mental  models)  may  be  thought  of  in  terms  of 
declarative  knowledge  (academic  knowledge  that  can  be  readily  taught)  and  tacit  knowledge  (practical 
knowledge  acquired  through  experience)  (Horvath  et  al.,  1994a,  1994b).  Acquisition  and  retention  of 
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tacit  knowledge  is  likely  quite  relevant  to  collective  performance,  and  there  have  been  extensive 
content  studies  of  different  aspects  of  tacit  knowledge  among  effective  managers  and  for  military 
leadership  (Horvath,  1994a,  1994b).  Further  work  may  well  exist  in  the  literature  that  examines 
acquisition  and  retention  of  tacit  knowledge  required  for  collective  tasks.  For  example,  where  team 
membership  is  stable,  relevant  tacit  knowledge  for  the  capabilities  of  other  team  members  may  be 
acquired  during  the  retention  interval  through  performance  on  other  tasks. 

3.4  Conclusion 

‘'Collectives  ’  may  be  thought  of  largely  in  terms  of  team  and  organizational  structures.  Collective 
tasks  can  be  considered  in  terms  of  individual  sub-tasks,  critical  sub-tasks,  interactive  sub-tasks  tasks, 
and  team  tasks,  with  the  relative  contribution  each  to  collective  performance  varying  with  the 
collective  task.  Uniquely  collective  skills  can  be  considered  in  terms  of  individual  interactive  skills 
and  team  work  skills  such  as  co-ordination,  communication  and  information  exchange  in  relation  to 
internalized  team  mental  models  and  awareness  of  internal  team  status  with  respect  to  sub-task 
performance,  and  as  such  can  be  seen  as  largely  cognitive.  This  suggests  that  decay  in  cognitive  sub¬ 
tasks  and  the  cognitive  aspects  of  collective  tasks  such  as  retention  of  knowledge  relating  to  different 
components  of  team  mental  models  should  be  considered  within  the  overall  framework  of  either 
individual  or  collective  skill  retention,  rather  than  as  distinct  and  separate  issue. 
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4.  Collective  Skill  Retention  and  Fading 

The  purpose  of  this  section  is  to  outline  some  general  issues  prior  to  a  more  detailed  discussion  in 
subsequent  sections  of  factors  likely  to  affect  retention  of  collective  performance,  and  existing  models 
for  predicting  retention. 

Skill  fading  can  be  described  as  the  measurable  decrement  in  performance  of  a  skill  in  relation  to  a 
criterion.  Skill  retention  has  been  defined  as  the  maintenance  of  skills  as  learned  behaviors  and 
procedures  over  long  periods  of  time  without  practice  (Schendel  &  Hagman,  1991).  Skills  fading  is 
thought  to  occur  because  perceptual,  motor,  or  cognitive  processes  that  underlie  skilled  performance 
decay  or  break  down  in  some  way.  This  may  be  because  a  subordinate  execution  skill  has  decayed  (such 
as  keyboarding),  or  memory  for  mental  model  for  the  rules  or  knowledge  governing  the  exercise  of  that 
component  skill  has  faded  or  become  inappropriate.  Alternatively,  the  skill  may  remain  intact  but 
become  inaccessible  to  the  individual  in  some  way.  Thus,  performance  on  some  sub-task  may  or  may 
not  reflect  actual  decay  in  the  underlying  knowledge  or  skill:  this  represents  a  considerable 
methodological  challenge  to  control,  for  instance,  the  effect  of  motivation  on  a  team  or  individual  during 
a  retention  test  and  to  interpret  the  results  of  retention  data  based  on  performance  data. 

Individual  skill  retention  is  often  described  using  a  power  function,  which  relates  performance  to  the 
interval  between  initial  training  and  subsequent  testing  (Shute  and  Gawlick,  1995).  This  function  is 
typically  seen  as  negatively  accelerated,  with  performance  declining  most  rapidly  soon  after  learning,  but 
at  lesser  rates  over  time.  Indeed,  most  learning  functions  (  acquisition,  transfer  of  training)  appear  best 
described  by  a  power  function  (Newell  and  Robinson  (1981;  cited  in  Swezey  and  Llaneras  (1997). 

Skill  loss  can  be  characterized  with  two  parameters.  The  slope  of  the  curve  describes  the  rate  at  which 
skill  is  lost.  The  asymptote  describes  the  level  at  which  performance  levels  out.  The  power  function 
offers  the  most  accurate  description  of  retention  for  a  range  of  tasks  and  materials  (Bryant  and  Angel, 
2000),  with  different  skills  showing  different  rates  of  decay,  and  some  military  skills  exhibiting  a  severe 
performance  decrement  in  as  little  as  two  weeks.  Generally  memory  for  knowledge  exhibits  the  most 
rapid  decay  and  continuous  perceptual  motor  skills  the  slowest  rate  (Wisher  et  al.,  1991). 

Performance 


Figure  5:  A  typical  retention  curve. 
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4.1  Skill  Development 

Anderson  (1995),  describes  skill  acquisition  as  occurring  in  3  sequential  stages. 

•  Cognitive.  An  individual  first  develops  knowledge  of  the  task  in  the  form  of  rules, 
procedures  and  constraints  called  declarative  knowledge.  Accessing  this  knowledge 
requires  conscious  recall  effort,  and,  if  retrieved  successfully,  this  knowledge  can  be  used 
to  guide  performance. 

•  Associative.  With  practice  and  knowledge  of  results,  the  individual  becomes  increasingly 
adept  in  associating  cues  with  task  steps.  These  associations  form  an  increasingly 
elaborated  system  of  if-then  rules  that  guide  behaviour. 

•  Autonomous.  Associations  that  have  formed  become  increasingly  automatic,  and  can  be 
retrieved  and  executed  with  little  conscious  effort.  Performance  may  be  thought  of  as  a 
procedural  response  to  situational  cues. 

Over  these  stages,  individuals  acquire  expertise  or  mental  models  of  domain  knowledge  in  terms  of 
content  (concepts,  cues,  procedures,  etc)  and  organization  for  retrieval  during  the  task.  This  theory  of 
skill  development  describes  the  process  that  an  individual  undergoes  during  skill  acquisition.  The 
question  is  whether  collective  or  team  or  team  performance  can  be  considered  in  terms  of  individual 
skills  jointly  exercised  or  must  be  conceptualized  in  a  different  way.  The  former  view  assumes  that 
individual  team  members  can  acquire  skills  such  as  co-ordination  and  information  exchange,  and  lends 
itself  to  study  of  individual  skill  retention  curves  for  interactive  skills  and  team  related  knowledge  in 
order  to  predict  retention  of  collective  performance. 

Although  we  were  not  able  to  identify  any  models  of  collective  skill  acquisition  as  such,  one  model  of 
skill  acquisition  for  teams  is  the  Team  Evolution  and  Maturation  (TEAM)  Research  Paradigm.  This 
model  proposes  that,  as  teams  mature,  there  are  two  different  but  parallel  “tracks”  of  skill  development 
(Morgan  et  al.,  1986;  cited  in  Swezey  and  Llaneras,  1997):  task  work  and  teamwork  (see  Figure  6). 

•  Task  work  skills  refer  to  an  individual’s  ability  to  perform  their  role  in  the  team. 

•  Teamwork  skills  refer  to  the  group’s  ability  to  perform  together  in  terms  of  information 
exchange,  overall  awareness,  and  adaptation. 

Taken  together  with  Roth’s  (1992)  somewhat  different  distinction  between  individual  skills  and 
collective  skills,  this  suggests  there  may  be  three  types  of  skill  to  be  distinguished.  Individual  skills  for 
the  one  person  tasks  in  the  team  (can  be  trained  as  an  individual).  Interactive  skills  required  by  the 
individual  to  perform  as  a  team  member  (can  be  partly  trained  as  an  individual  in  isolation  from  any 
team  task).  Team  skills  used  only  in  the  context  of  a  particular  team  and  a  particular  task  (cannot  be 
learned  in  isolation  from  other  members  of  the  team). 
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Figure  6:  Team  evolution  model  Adapted  from  the  TEAM  model  (Morgan  et  al  1986) 

The  TEAM  model  applies  to  skill  development  not  skill  retention  but  it  seems  likely  that  retention  of 
collective  perfonnance  will  depend  on  the  stage  of  team  evolution  reached  when  measurement  of 
retention  starts.  There  is  also  the  issue  of  what  to  measure  and  the  need  to  develop  effective  measures 
for  interactive  skills  and  team  skills  in  addition  to  individual  skills.  One  implication  of  the  model  is  that 
team  evolution  is  related  not  only  to  team  membership  but  to  particular  tasks.  This  raises  the  question 
of  transfer  of  team  work  skills  from  one  task  to  another,  with  the  answer  likely  dependent  on  the  degree 
of  similarity  between  the  two  tasks,  combined  in  some  way  with  the  level  of  maturation  of  the  team 
with  respect  to  other  tasks. 


4.2  Retention  of  Individual  Skills 

Considerable  work  has  examined  how  different  performance  on  individual  tasks  is  maintained  over 
time.  A  recent  review  by  Sabol  and  Wisher  (2001),  for  example,  considers  the  current  state  of  skill 
decay  research  with  respect  to  individual  military  tasks  and  distinguishes  between  knowledge, 
decision  and  execution  skills  present  to  differing  degrees  in  all  tasks. 

•  Knowledge-based  skills  fade  substantially  in  the  first  few  months  after  training,  but  the 
degree  of  fading  depends  on  how  that  knowledge  is  measured,  and  whether  the  test  demands 
recognition  or  recall  (Wisher  et  al.,  1999).  Within  knowledge  based  tasks,  memory  for 
complex  procedural  tasks  is  likely  to  decay  more  quickly  (Schendel  and  Hagman,  1991; 
Sabol  and  Wisher,  2001). 

•  Cognitive  or  decision-based  skills  including  problem  solving  show  moderate  rates  of  decay. 
Work  by  Semb  and  Ellis  (1994;  reported  in  Sabol  and  Wisher,  2001)  summarising  more 
than  20  studies  with  both  military  and  non-military  tasks,  suggests  that  these  decay  in  the 
region  of  20%  over  periods  of  3-6  months  or  longer. 
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•  Execution  skills  or  perceptual-motor  skills  vary  in  their  decay.  Rates  of  decay  depend 
greatly  on  whether  the  skill  is  continuous  (having  no  clear  beginning  or  end,  such  as 
driving)  or  discrete  (having  a  distinct  start  and  end  point,  such  as  assembling  a  machine 
i.e.  procedural  tasks).  Skills  that  are  continuous  have  shown  extremely  low  levels  of 
decay  even  after  two  years  without  practice  (Childs  and  Spears,  1986).  Discrete  or 
procedural  skills  often  show  marked  decay  in  a  few  weeks. 

However,  the  time  span  over  which  retention  has  been  studied  is  typically  short.  Moreover,  rates  of 
decay  vary  widely,  and  the  type  of  tasks  studied  and  the  manner  of  measurement  in  acquisition  and 
retention  studies  often  are  unrepresentative  of  real  world  tasks,  or  use  subjects  with  limited  experience 
of  the  task  in  question  (Swezey  and  Llaneras,  1997). 

Bryant  and  Angel  (2000)  focused  on  individual  skill  retention  and  focused  on  models  of  skill 
fading/retention  and  tools  for  determining  when  refresher  training  is  required  in  the  military  and 
examined  eight  factors  shown  to  affect  skill  retention: 

•  Retention  Interval. 

•  Opportunity  to  practice. 

•  Degree  of  initial  learning  or  over-learning. 

•  Method  of  training. 

•  Similarity  of  training  and  performance  environments. 

•  Type  of  task 

•  Method  of  testing. 

•  Individual  differences. 

Several  classes  of  models  for  predicting  skill  retention  were  identified. 

•  Subjective  approaches  involve  some  form  of  self-assessment  of  retention  and/or  the  need 
for  refresher  training  on  the  part  of  trained  individuals.  These  techniques  may  yield 
accurate  estimates  of  retention  but  tend  to  be  consistently  optimistic  and  tend  to  be 
mistrusted  as  a  predictive  tool. 

•  Qualitative  approaches  indicate  categorically  how  skills  fade  over  time  in  relation  to 
certain  key  factors.  These  models,  as  they  stand  to  date,  are  more  proscriptive  than 
predictive  (offering  strategies  to  reduce  skill  loss  rather  than  means  to  predict  rate  of  skill 
loss)  and  do  not  solve  the  practical  problems  of  predicting  when  proficiency  of  skills  will 
decline  below  a  criterion  or  when  refresher  training  will  be  needed  but  they  do  offer 
approaches  that  may  be  developed  into  quantitative  models  in  the  future. 

•  Quantitative  approaches  -  The  best  developed  model  of  skill  retention  found  by  Bryant 
and  Angel  (2000)  was  the  U.S.  Army  Research  Institute's  Users'  Decision  Aid  (UDA) 
model,  developed  specifically  to  provide  quantitative  predictions  of  skill  retention  for 
military  tasks.  The  UDA  model  is  more  applicable  to  the  prediction  of  procedural  and 
perceptual-motor  abilities  than  cognitive  abilities  and  is  described  in  some  detail  in  a 
following  section. 


4.3  Conceptualising  Collective  Skill  Retention 

Collectives  or  teams  and  their  tasks  can  vary  in  several  ways  that  may  interact  with  skill  retention. 
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•  The  interdependence  of  organizational  structure  (e.g.  the  division  of  specialty  skills 
among  collective  members  and  sub-groups). 

•  The  number  of  levels  within  the  organization. 

•  Interdependence  of  tasks  performed  by  team  members  or  sub-groups  to  complete  the 
collective  task  in  question.  This  interdependence  may  be  in  terms  of  the  need  to  exchange 
information,  or  in  terms  of  the  criticality  of  sub-tasks  with  respect  to  speed,  accuracy  or 
timeliness  of  performance. 

•  The  balance  of  individual,  interactive,  and  teamwork  skills  required  for  the  task. 

•  The  balance  between  knowledge,  decision  and  execution  skills  required  for  the  task. 

•  The  period  of  time  over  which  the  team  or  collective  has  worked  together 

•  The  types  of  events  experienced  during  the  retention  interval  and  their  relevance  to  the 
collective  task  of  interest. 

•  The  degree  of  evolution  as  a  team  for  the  task  in  question. 

•  The  degree  of  membership  change  over  the  retention  interval. 

Some  collective  tasks  depend  primarily  on  sub-tasks  requiring  little  interaction  or  co-ordination 
between  collective  members.  (For  example  a  group  of  phone  operators.)  In  such  cases,  the  overall 
performance  of  the  collective  is  primarily  an  aggregate  of  individual  performance.  (More  phone 
operators  result  in  more  phone  calls  handled.)  Other  collective  tasks  rely  heavily  on  co-ordinated  sub¬ 
tasks  among  several  people.  In  turn,  the  tasks  to  be  co-ordinated  may  be  virtually  identical  (a  team  of 
rowers)  or  differentiated  (a  tank  crew).  In  each  case  the  balance  of  skill  types  will  vary.  As  such,  the 
“collectivity”  of  any  task  will  vary  along  the  dimension  of  how  much  interaction  and/or  co-ordination 
is  required  to  complete  the  task. 

Consideration  of  the  issues  outlined  above  leads  to  the  following  observations. 

•  For  tasks  where  collective  members  perform  primarily  as  individuals,  with  less  need  for 
interdependence  or  interaction  among  sub-tasks,  decay  can  be  conceptualized  as  some 
aggregate  of  the  rate  for  individual  sub-task  skills.  With  little  co-ordination  required,  the 
rate  of  decay  in  the  overall  task  of  the  collective  will  relate  most  closely  performance  on 
individual  tasks,  not  their  co-ordination. 

Thus,  for  tasks  in  which  collective  members  function  as  relatively  independent  agents,  the 
best  predictor  of  collective  skill  retention  may  be  the  retention  curves  for  individual  sub¬ 
tasks,  which  in  turn  will  depend  on  the  type  of  sub-task  involved:  knowledge  based, 
decision  based,  perceptual  motor.  Prediction  might  need  to  be  modified  by  the  degree  to 
which  resources  can  be  re-allocated  within  the  collective  to  compensate  for  differences  in 
rates  of  decay  among  individual  collective  members. 

•  The  greater  the  interaction  required  among  interdependent  and  critical  collective  sub¬ 
tasks,  the  more  retention  of  performance  will  depend  on  the  rate  of  decay  of  individual 
interactive  skills  and  group  team  work  skills,  in  addition  to  the  rate  of  decay  of  individual 
task  work  skills. 

•  The  more  specialized  the  skills  among  individuals  (or  sub-units)  in  the  collective,  the 
greater  will  be  the  interdependence  of  the  critical  sub-tasks  they  perform.  Thus,  as  the 
level  of  specialization  rises,  the  greater  will  be  the  dependence  of  the  overall  collective 
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task  on  decay  of  specialized  task  work  skills.  Conversely,  the  more  it  can  be  generalized 
the  skills  among  collective  members  (perhaps  as  a  result  of  more  cross  training,  or  for  a 
system  with  sub-tasks  based  on  more  generic  transportable  skills),  the  less  dependent  will 
overall  collective  performance  be  on  critical  sub-tasks. 

•  With  a  rise  in  the  number  horizontal  and  vertical  subdivisions  in  the  organization  (e.g.  the 
more  teams  it  contains),  dependence  on  team  work  skills  will  increase,  especially  with 
respect  to  information  exchange  and  awareness  among  team  members  of  progress  on  critical 
tasks  (and  among  team  leaders,  progress  of  other  teams). 


4.4  Retention  of  Collective  Skills 

Most  of  the  foregoing  observations  are  based  on  extrapolation  for  little  research  was  found  that 
directly  addresses  the  issue  of  how  long  performance  on  collective  or  team  tasks  is  likely  to  be 
retained.  Roth  (1992)  notes  that  although  a  considerable  amount  is  known  about  the  effects  of  time 
without  training  on  individual  task  performance,  “Essentially  no  previous  work  on  skill  decay  in 
collective  tasks  has  been  done  for  real-world  tasks.”  Even  work  directly  intended  to  explore  tank 
gunnery  skills,  for  example,  concluded  with  regret  that,  in  the  end,  there  is  little  if  any  research  which 
focuses  on  the  collective  rather  than  the  individual  level  (Rowatt  and  Shlechter,  1992). 

The  only  available  work  on  collective  skill  retention  suggests  that  collective  task  proficiency  is 
negatively  affected  by  a  retention  interval  of  less  than  one  month  (Roth,  1992).  To  assess  this,  Roth 
compared  performance  scores  over  a  period  of  one  month  for  several  Light  Infantry  platoons  on 
fourteen  collective  tasks,  such  as  employing  fire  support,  occupying  a  patrol  base,  performing  aerial 
re-supply  and  moving  tactically.  The  original  measures  were  collected  after  training  conducted  by  the 
platoon’s  parent  division  and  represented  the  number  of  sub-tasks  rated  as  acceptable  according  to 
training  evaluation  standards.  The  second  measure,  collected  one  month  later,  was  done  during 
training  at  the  Joint  Readiness  Training  Centre  (JRTC),  using  the  same  measures,  apparently 
completed,  in  both  cases,  by  the  unit  commanders. 


Task  Title 

Number  of 
Platoons 
Scored  on 
the  Task 

External 

Evaluation 

Performance 

Score 

JRTC 

Performance 

Score 

Performance 

Change 

in 

One  Month 

1 .  Infiltrate/Ex-filtrate 

5 

.53 

.93 

+.40 

2.  Perform  Aerial  Re-supply 

2 

.57 

.59 

+.02 

3.  Sustain 

4 

.75 

.81 

+.06 

4.  Perform  a  Passage  of  Lines 

1 

1.00 

1.00 

0 

5.  Move  Tactically 

9 

.67 

.63 

-.04 

6.  Occupy  a  Patrol  Base 

1 

.88 

.75 

-.06 

7.  Consolidate  and  Reorganize 

6 

.67 

.50 

-.17 

8.  Occupy  an  Assembly  Area 

7 

.84 

.67 

-.17 

9.  Employ  Fire  Support 

4 

.90 

.68 

-.21 

10.  Perform  Helicopter  Movement 

1 

.96 

.63 

-.23 

11.  Perform  Link-Up 

3 

.83 

.58 

-.25 

12.  Assault 

1 

.75 

.50 

-.25 

13.  Occupy  OP/Perform  Surveillance 

2 

.88 

.60 

-.28 

14.  Perform  Tactical  Road  March 

1 

.92 

.57 

-.34 

Means 

- 

.78 

.67 

-.11 

Table  2:  Collective  task  performance  scores  over  a  one-month  interval  (Roth,  1992) 
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As  the  table  shows,  while  some  of  the  collective  tasks  deteriorated  over  the  month,  with  the  average 
decrement  of  about  1 1%,  six  out  of  fourteen  improved  or  showed  little  change.  Of  those  that  did 
decay,  the  average  performance  decrement  was  closer  to  25%.  There  was  no  analysis  of  the  collective 
tasks  in  terms  of  their  need  for  interaction  or  the  interdependence  of  the  sub-tasks.  The  variety  of 
tasks  performed,  the  wide  fluctuations  in  the  sample  sizes  for  different  tasks  and  the  fact  that  the 
evaluation  was  conducted  by  the  unit  commanders  call  into  question  the  reliability  of  the  data. 
Nevertheless,  as  Roth  notes,  the  average  performance  deterioration  is  similar  to  the  rates  evidenced  for 
individual  tasks  shown  to  be  fairly  resistant  to  skill  decay  (Rose  et  al.,  1995a). 


Human systems®  Incorporated 


Collective  and  Cognitive  Skill  Retention 


Page  21 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


Page  22 


Collective  and  Cognitive  Skill  Retention 


Y\umdx\systems  Incorporated 


5 


Predicting  Collective  Skill  Retention 


The  purpose  of  this  section  is  to  explore  factors  that  may  affect  retention  of  performance  on  collective 
tasks.  We  found  little  research  and  theory  that  discusses  the  prediction  and  retention  of  collective 
performance  as  such.  This  likelihood  was  recognized  from  the  start  and,  consequently,  much  of  this 
discussion  is  speculation  based  largely  on  extrapolation  from  the  literature  on  individual  skill  retention 
and  team  work,  and  intended  to  foster  debate  and  serve  as  a  point  of  departure. 2 

Performance  of  collective  tasks  depends,  in  differing  degrees  for  different  collective  tasks,  on 
individual  skills  for  sub-tasks  on  the  critical  path  for  collective  performance  and  on  individual 
interactive  skills.  Retention  for  such  skills  is  likely  to  follow  a  similar  pattern  as  individual  skill 
retention.  In  general,  we  believe  that  many  of  the  same  factors  are  likely  to  influence  skill  retention  at 
a  collective  level  to  the  degree  that  similar  tasks  are  present,  and  on  the  critical  path  for  the  overall 
collective  task. 

For  retention  at  a  collective  level,  however,  the  overall  unit  of  analysis  is  no  longer  the  individual,  but 
the  whole  group  or  organization.  While  it  is  important  to  consider  the  impact  of  factors  already 
implicated  in  individual  skill  retention  at  the  collective  level,  we  have  also  considered  features  of  the 
collective  unit  likely  to  impact  on  collective  skill  retention. 

Training  at  the  collective  level  appears  unique  in  that,  especially  for  larger  collectives,  it  may  have  no 
distinct  start  and  end  point  from  which  to  measure  skill  decay  for  the  collective  as  a  whole.  How,  for 
example,  does  one  establish  a  retention  curve  for  collective  tasks  performed  at  the  level  of  a  battle 
group  based  on  a  battalion,  or  even  a  combat  team  based  on  an  infantry  company.  Instances  of 
training  for  larger  collectives  may  be  spaced  apart  by  as  much  as  a  year  in  the  annual  training  cycle. 

The  circumstances  of  that  training  may  vary  widely  in  terms  of  current  team  membership,  the  level  of 
evolution  of  the  team(s)  in  the  collective,  the  exact  nature  of  the  collective  task,  and  the  circumstances 
in  which  it  is  undertaken.  Test  re-test  data  may  have  to  be  collected  under  very  different 
circumstances,  making  measurement  of  decay  under  controlled  comparable  conditions  difficult  or 
even  impossible.  On  the  other  hand,  individual  skills  for  independent  sub-tasks  and,  to  a  lesser  extent, 
interactive  skills  possessed  at  the  individual  level,  may  be  trained  and  assessed  at  the  individual  level  - 
making  study  more  practicable. 

(In  the  search  for  practical  methodologies  and  suitable  collective  tasks  and  teams  to  study,  there  may  be 
merit  in  examining  team  training  simulators  used  in  different  services.  Such  an  example  might  be  the 
Operations  Room  Team  Trainer  recently  brought  into  sendee  for  the  HALIFAX  class  frigate.  This 
trainer  is  capable  of  conducting  watch  and  watch  about  team  training  simultaneously  for  two  established 
ship ’s  operation  room  teams  (approximately  30  persons)  over  prolonged  periods.  It  also  provides 
sophisticated  data  capture  and  archiving  capabilities  that  would  permit  comparison  of  team  performance 
across  different  intervals  and  conditions.  Once  common  dimensions  likely  of  fundamental  interest  to 
collective  performance  have  been  identified,  such  fundamentals  could  be  examined  across  service 
boundaries.) 

This  section  discusses  the  sets  of  factors  in  the  following  order: 

•  Features  of  the  collective  task 


2  ...  . 

A  previous  literature  review  examined  factors  known  to  influence  individual  skill  retention  (Bryant  and  Angel,  2000)  and  the  reader  is 
referred  to  that  work. 
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•  Features  of  the  collective  itself 

•  Characteristics  of  individuals  forming  the  collective 

•  Features  of  the  training  received  by  the  collective 

•  Characteristics  of  the  retention  interval. 


5.1  Features  of  the  Collective  Task 

The  literature  on  individual  skill  retention  suggests  several  task  features  likely  to  influence  skill 
retention.  These  include,  for  example,  the  number  of  steps  in  the  task,  whether  the  steps  of  the  task 
must  be  performed  in  a  set  sequence,  and  the  existence  of  memory  aids  to  perform  the  task  (Bryant 
and  Angel,  2000).  Many  such  task  features  are  present,  to  some  degree,  in  collective  tasks  and 
performance  is  likely  to  be  retained  in  much  the  same  way. 

The  section  covers  the  following  features  of  the  collective  task: 

•  The  number  of  collective  tasks  assigned  to  the  collective  unit. 

•  The  number  of  steps  in  individual  sub-tasks, 

•  The  availability  of  interstep  cueing  within  sub-tasks; 

•  The  predictability  of  sub-task  procedures  and  conditions  of  performance; 

•  The  number  of  discrete  skills  required  by  the  collective  task; 

•  The  number  of  sub-units  needed  to  perform  the  collective  task; 

•  Interchangeability  among  sub-tasks; 

•  Interdependence  among  sub-tasks; 

•  The  potential  for  error  compensation; 

•  The  need  for  information  exchange  among  collective  members; 

•  The  cognitive  complexity  of  sub-tasks. 

5.1.1  Number  of  collective  tasks  performed  by  the  collective 

Some  collectives  focus  on  only  one  collective  task  in  one  setting.  Many  military  collectives  may  be 
required  to  perform  many  different  tasks  under  a  wide  variety  of  conditions.  The  variety  and 
differences  among  the  collective  tasks  a  given  collective,  such  as  a  platoon,  or  ship,  may  be  required  to 
perform  will  affect  the  amount  of  knowledge  and  number  of  procedures  and  skills  the  members  of  the 
collective  have  to  learn  and  retain. 

Depending  on  the  overlap  between  them,  each  different  collective  task  may  require  a  different  skill  set. 
This  will  increase  the  procedural  and  other  knowledge  required.  There  may  also  be  longer  gaps 
between  opportunities  to  practice  given  tasks,  and  therefore  longer  retention  intervals.  For  such 
reasons,  Roth  (1992)  argues  that  collectives  that  perform  more  tasks  are  likely  to  show  higher  levels  of 
absolute  skill  loss  than  collectives  that  perform  fewer  tasks.  However,  there  is  another  way  to  look  at 
this:  in  terms  of  transfer  of  training  among  the  different  collective  tasks.  Transfer  of  training  may  be 
positive  or  negative  depending  on  the  degree  of  similarity  or  commonality  among  the  sub-tasks 
required  and  the  circumstances  under  which  they  must  be  performed.  For  instance,  a  collective 
member  may  be  required  to  drive  for  all  collective  tasks,  or  to  drive  for  one  collective  task  and 
perform  some  other  sub-task  in  another  collective  task. 
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The  degree  of  similarity  between  these  collective  tasks  and  their  component  sub-tasks  will  affect  the 
transfer  of  training  as  result  of  practice  on  different  tasks  during  the  retention  interval.  If  the  other 
collective  tasks  or  their  sub-tasks  are  similar  then  the  transfer  of  training  will  improve  retention  of  the 
collective  task  of  interest.  Depending  on  the  degree  of  dissimilarity,  a  negative  transfer  effect  can  be 
expected,  and  retention  reduced. 

As  the  number  of  dissimilar  collective  tasks  assigned  to  the  unit  increases,  the  probability  that  its 
structure  must  be  re-arranged  for  each  task  increases.  This  will  add  to  the  amount  of  information  and 
differences  in  procedures  that  collective  members  individually  and  as  a  whole,  must  master  and 
remember. 

This  sort  of  analysis  may  need  to  be  repeated  for  the  sub-tasks  undertaken  within  each  collective  task. 
Predictions  for  retention  of  collective  performance: 

•  The  effect  on  retention  of  the  number  of  collective  tasks  assigned  to  a  unit  will  depend  on  the 
degree  of  transfer  between  sub-tasks. 

5.1 .2  The  number  of  steps  in  individual  sub-tasks 

Disregarding  the  need  for  interaction  with  others,  for  individual  sub-tasks,  the  higher  the  number  of 
steps  in  the  task,  the  more  complex  the  procedure  and  the  greater  the  load  on  memory,  and  the  less 
likely  task  performance  is  to  be  retained  (e.g.  Rose  et  al.,  1985a). 

This  may  affect  collective  performance  in  two  ways.  First,  as  the  number  of  sub-tasks  to  be  completed 
by  the  collective  or  team  increases,  the  faster  overall  collective  performance  is  likely  to  decay. 

Second,  the  greater  the  number  of  steps  required  by  individual  sub-tasks  on  the  critical  path  for  overall 
collective  performance,  the  greater  the  likely  effect  of  the  faster  decay  of  those  sub-tasks  on  the 
collective  task  as  a  whole. 

This  factor  is  likely  to  interact  with  the  size  and  diversity  of  specialities  within  the  collective.  Large 
and  more  diversified  collectives  are  likely  to  require  more  co-ordination.  Some  aspects  of  co¬ 
ordination  may  be  assigned  explicitly  to  one  or  more  members  of  the  collective  as  a  specialized 
command  and  control  function.  In  other  cases,  co-ordination  functions  may  be  left  to  individual 
collective  members  to  dovetail  with  their  own  tasks,  adding  steps  (and  possibly  requiring  multi¬ 
tasking)  to  their  existing  tasks,  thereby  increasing  the  complexity  of  the  procedures  and  the  load  on 
memory,  and  with  that,  the  rate  of  decay. 

Predictions  for  collective  skill  retention: 

•  The  greater  the  number  of  steps  in  a  sub-task,  the  faster  the  rate  of  decay. 

•  The  more  critical  the  sub-task,  the  greater  the  effect  on  collective  performance. 

5.1.3  Availability  of  interstep  cueing  for  sub-tasks 

Individual  sub  tasks  differ  in  the  degree  to  which  they  provide  feedback  on  performance  of  the  past 
step  and  cueing  as  to  the  next  step3.  The  training  literature  shows  that  in  highly  complex,  or  very 
procedural  and/or  lengthy  tasks,  interstep  cueing  can  help  individuals  to  first  learn  and  then  retain 
knowledge  and  execute  tasks  promptly  and  accurately  (Rose  et  al.,  1985a).  For  example,  during 


3  Feedback  in  this  context  is  a  specific  product  of  the  task,  and  stands  in  contrast  to  feedback  received  during  training  which  is  discussed  in 
the  section  on  training. 
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mechanized  mine  breaching  operation,  the  step  by  step  lane  clearance  task  is  a  highly  choreographed 
procedure. 

However,  the  benefits  of  feedback  and  cueing  may  diminish  as  individuals  progress  in  status  from 
novice  to  expert.  The  experienced  performer  may  actually  find  that  some  aspects  of  feedback  and 
cueing  may  interfere  with  task  performance,  and  are  better  discarded  or  reduced.  Conversely, 
selective  and  temporary  re-introduction  of  feedback  and  cueing  after  a  period  away  from  the  task  may 
assist  recollection. 

Within  collectives,  interstep  cueing  is  potentially  even  more  important  to  the  prediction  of  skill 
retention.  With  different  people  working  on  different  aspects  of  the  overall  collective  task,  feedback 
about  the  correctness  of  past  steps  and  prompts  for  future  steps  are  likely  to  help  all  members  of  the 
collective  co-ordinate  among  their  sub-tasks  and,  especially  collective  members  responsible  for  co¬ 
ordinating  others,  to  gauge  how  well  the  overall  collective  task  is  progressing.  The  greatest  benefit  is 
likely  for  sub-tasks  on  the  critical  path  for  the  collective  task  in  question. 

Predictions  for  retention  of  collective  performance: 

•  Interstep  cueing  for  critical  and  interdependent  tasks  will  improve  retention. 

5.1.4  The  predictability  of  sub-task  procedures  and  context 

For  individual  skill  retention  more  procedural  tasks  are  more  resistant  to  skills  fading  (Sabol  and 
Wisher,  2001).  This  may  be  because  highly  procedural  tasks  are  more  likely  to  provide  consistent 
cues  to  their  sequence  of  completion.  This  effect  may  relate  partly  to  the  similarity  of  the  context  in 
which  the  task  is  performed  -  cues  may  be  more  difficult  to  recognize  in  widely  differing 
circumstances.  (This  effect  probably  acts  in  much  the  same  way  that  positive  transfer  of  training  is 
greater  under  more  equivalent  conditions.) 

Tasks  that  are  less  procedural  or  that  cannot  be  made  procedural  require  a  more  complex  set  of  rules  to 
guide  performance:  rules  that  are  more  likely  to  be  forgotten.  In  collective  work,  for  less  procedural 
sub-tasks,  it  may  also  be  more  difficult  to  recognize  when  one  sub-  task  is  completed  satisfactorily  and 
the  next,  dependent,  sub-task  may  start.  This  would  make  co-ordination  more  challenging. 

Predictions  for  collective  skill  retention: 

•  Consistent  and  comprehensive  procedures  will  improve  retention. 

•  Consistent  circumstances  for  performance  will  improve  retention. 

5.1.5  The  number  of  discrete  skills  required  for  the  collective  task 

As  the  number  of  discrete  skills  needed  to  complete  a  collective  task  increases,  there  will  be  an  increase 
in  the  amount  of  information  to  be  learned  about  how  each  individual  sub-task  intersects  with  others  - 
and  in  the  amount  and  quality  of  information  that  must  be  exchanged  and  comprehended  across 
specialist  boundaries  (individual  or  team)  within  the  collective.  This  increase  requirement  for 
knowledge  and  memory  is  likely  to  make  retention  of  the  collective  task  as  a  whole  more  difficult. 

The  higher  the  level  of  the  collective  within  an  organization  (e.g.  the  command  team  for  an  battle  group 
versus  the  command  team  for  a  infantry  platoon),  the  more  likely  there  is  to  be  diversity  of  skills 
required,  and  more  specialized  the  skills.  As  the  diversity  and  level  of  specialisation  among  diverse 
skills  increases  in  a  group,  then  it  is  likely  that  co-ordination  and  co-operation  will  be  more  difficult  to 
achieve,  and  collective  performance  will  be  faster  to  decay. 
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Predictions  for  retention  of  collective  performance: 

•  As  diversity  of  among  specialists  (sub-groups  or  individuals)  increases  within  a  collective, 
rates  of  decay  will  increase. 

•  The  higher  the  level  of  the  collective  in  the  organization,  the  greater  the  diversity  and  level 
of  specialisation  required,  and  the  faster  the  decay. 

5.1.6  The  number  of  sub-units  needed  for  the  collective  task 

Collectives  tasks  vary  in  the  number  of  sub-units  or  teams  involved  and  the  number  of  levels  at  which 
the  sub-units  are  organized.  Commonly,  for  example,  several  different  tank  crews  must  work  together 
rather  than  as  individual  units,  organized  at  several  levels  (e.g.  tank  crew,  troop,  squadron,  regiment). 
With  more  sub-units,  more  co-ordination  will  be  required  and  more  relationships  among  sub-units  will 
need  to  be  considered  and  understood.  This  will  require  substantial  changes  in  how  sub-units  perform. 
Rather  than  working  as  individual  units,  for  example,  teams  must  now  work  to  ensure  that  information 
is  exchanged  with  other  sub-units.  This  issue  is  discussed  in  more  detail  in  section  6.1.10. 

Predictions  for  collective  skill  retention: 

•  More  sub-units  will  increase  the  rate  of  decay. 

•  More  organizational  levels  will  increase  the  rate  of  decay. 

5.1.7  Interchangeability  among  sub-tasks 

Sub-tasks  assigned  to  individuals  vary  in  the  extent  to  which  they  can  be  performed  by  different 
members  of  a  collective.  The  skills,  knowledge  or  procedures  required  to  perform  a  given  sub-task 
may  be  specialized  or  possessed  by  only  a  few  members  or  one  team  within  the  collective. 

The  extent  to  which  a  collective  sub-task  can  be  completed  by  several  members  of  the  team  or 
collective  is  likely  to  improve  the  collective’s  ability  to  maintain  overall  collective  performance  for 
several  reasons.  If  one  member  is  unable  to  perform  the  task,  other  team  members  may  be  able  to  take 
over  the  task,  or  assist  the  individual  responsible.  For  example,  although  artillery  Forward 
Observation  Officers  are  specially  trained  to  call  for  and  adjust  artillery  fire,  other  members  of  a 
combat  team  can  also  do  this  if  needed.  The  degree  to  which  collective  members  are  cross  trained  in 
others  sub-tasks  and/or  the  level  of  similarity  between  tasks  will  affect  this  capability. 

Part  of  this  capability  requires  a  shared  mental  model  of  the  tasks  of  others  (Mathieu  et  al.,  2000). 
Insight  into  the  sub-tasks  of  others  is  not  only  likely  to  facilitate  dovetailing  among  interdependent 
individual  sub-tasks  but,  by  distributing  memory  for  common  sub-task  knowledge  and  procedures 
among  different  collective  members,  overall  retention  within  the  collective  as  a  whole  is  likely  to 
improve  with  the  possibility  that  one  collective  member  can  prompt  others  when  they  forget,  or  even 
substitute  for  them  entirely.  Having  a  shared  mental  model  of  collective  sub-tasks  can  be  expected 
improve  the  ability  of  the  collective  as  a  whole  to  complete  the  task  and  improve  collective  skill 
retention. 

A  related  point  is  that,  regardless  of  the  potential  for  interchanging  team  members  among  sub-tasks, 
actually  using  that  potential  will  depend  on  awareness  of  the  progress  and  needs  of  others  in  the 
collective,  and  co-ordination  of  any  interchanges. 

Predictions  for  collective  skill  retention: 

•  A  greater  proportion  of  interchangeable  sub-tasks  will  increase  retention. 
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•  Better  awareness  among  collective  members  of  progress  among  interchangeable  sub-tasks 
will  improve  retention. 

5.1.8  Interdependence  among  sub-tasks 

The  extent  to  which  the  success  of  collective  tasks  require  co-ordination  among  team  members  to 
complete  their  sub-tasks,  or  for  completed  sub-tasks  to  be  dovetailed  in  some  way,  has  been  cited  as  a 
crucial  factor  for  overall  success  or  failure  of  collective  tasks  (Dieterley,  1988;  Swezey  and  Llaneras, 
1997).  Decay  in  performance  of  interdependent  tasks  is  likely  to  have  a  marked  effect  on  how  well 
overall  performance  of  the  collective  task  is  retained.  Greater  interdependence  among  sub-tasks  is 
likely  to  be  associated  with  faster  decay  in  the  overall  collective  task. 

Interdependence  has  a  number  of  dimensions.  It  may  be,  for  example,  that  the  performance  of  a 
collective  task  is  wholly  dependent  on  several  members  of  the  team  to  make  a  partial  contribution 
toward  the  completion  of  the  task.  For  a  tank  crew,  for  example,  successful  engagement  depends  on  the 
tactics  of  the  commander;  on  the  loader  loading  the  proper  ammunition,  on  the  gunner  sighting  the 
target,  and  on  the  driver  manoeuvring  appropriately.  If  part  of  any  component  task  is  not  completed,  the 
performance  of  the  collective  task  as  a  whole  may  fail.  In  this  example,  the  type  of  dependence 
combines  linear  and  parallel  dependencies,  as  most  sub-tasks  can  only  be  completed  after  the  previous 
sub-task  has  been  completed,  but  some  can  completed  in  parallel. 

For  less  interdependent  collective  tasks,  each  member  of  the  collective  may  need  to  complete  his  own 
sub-task  for  the  task  of  the  collective  as  a  whole  to  be  completed,  but  an  individual’s  ability  to  complete 
his  own  task  may  not  be  dependent  on  another  person  having  completed  their  sub-task.  An  example 
might  be  the  preparation  of  a  defensive  position  -  where  individual  fire  positions  all  have  to  be 
completed,  but  the  completion  of  one  fire  position  may  not  depend  on  the  completion  of  any  other. 

Criticality  of  the  interdependent  sub-tasks  may  also  vary.  Some  sub-tasks  may  require  continual  split 
second  timing  and  co-ordination  -  others  may  be  less  time  sensitive.  Criticality  may  also  vary  with 
respect  to  the  quality  of  completion  of  different  sub-tasks. 

A  task  may  also  be  performed  in  serial  or  in  parallel,  in  relation  to  the  structure  of  the  team  or 
organization.  Paris,  Salas,  and  Cannon-Bowers  (2000)  focus  on  the  dependency  between  tasks  and  the 
people  performing  the  tasks,  and  tasks  are  seen  as  being  performed  independently,  sequentially,  or  in 
parallel.  When  task/operator  dependency  is  very  high,  inadequate  performance  by  one  person  can 
greatly  influence  the  ability  of  another  person  at  a  different  station  to  complete  their  task.  Teams  in 
which  task  performance  is  organized  serially  may  be  compromised,  because  team  performance  will  be 
determined  by  the  strength  of  the  weakest  link  in  the  serial  chain.  Analysing  and  studying  such 
complexities  would  require  a  detailed  team  task  analysis,  and  comparisons  among  many  different 
aspects  of  interdependency  within  collective  structure  and  collective  tasks  would  probably  be 
facilitated  by  the  use  of  software  modelling  programs  such  as  Micro-Saint. 

The  number  of  dependencies  among  sub-tasks  and  the  need  for  interaction  related  for  their  completion 
will  vary  for  different  collective  tasks  and  different  function  allocations  within  collective  structures. 
Furthermore,  each  dependency  has  a  differing  potential  to  disrupt  the  ability  of  the  collective  to  perform 
the  task.  Recognizing  this,  Dieterly  (1988)  proposes  calculation  of  separate  “ Interdependence”  and 
“Interaction  ”  ratios  for  team  tasks  based  on  a  collective  or  team  task  analysis.  An  interaction  ratio 
would  identify  the  proportion  among  of  all  tasks  necessary  for  goal  achievement  (i.e.  on  the  critical 
path)  assigned  to  individuals  and  dependent  on  the  completion  of  other  tasks  by  other  individuals.  An 
interdependence  ratio  would  calculate  the  total  number  of  tasks  in  relation  to  the  maximum  number  of 
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sub-  tasks  that  could  be  handled  by  one  team  member,  in  parallel  or  sequentially  within  a  given  period 
of  time  for  completion  of  the  overall  collective  task. 

However,  while  such  ratios  may  go  some  way  to  deal  with  the  frequency  of  interaction  or  the  number 
of  dependencies  among  sub-tasks,  qualitative  aspects  of  those  interactions  or  dependencies  would 
remain  to  be  addressed.  Furthermore,  the  level  of  effort  required  to  calculate  such  ratios  is  likely  to  be 
high,  even  for  quite  small  collectives  performing  quite  simple  tasks.  The  most  efficient  way  in  which 
to  generate  such  indices  is  probably  to  conduct  this  analysis  as  part  of  system  design.  Indeed,  Dieterly 
advocates  such  an  approach  as  part  of  the  process  of  optimizing  function  allocation  within  teams 
maximize  team  performance  and,  subsequently,  facilitate  team  training. 

Dealing  specifically  with  collectives  Roth  (1992)  defines  performance  of  collective  tasks  as  being 
either  co-active  or  interactive.  Co-active  tasks  are  those  in  which  several  collective  members  perform 
the  same  or  similar  tasks  under  relatively  centralized  direction.  Interactive  tasks  require  different 
collective  members  to  co-ordinate  substantively  different  tasks  with  other  members  of  the  collective. 

Roth  argues  that  interactive  collective  tasks  require  more  effort  to  learn  co-ordination  and  division  of 
responsibilities,  and  so  are  less  likely  to  be  retained  than  co-active  tasks. 

Predictions  for  collective  skill  retention: 

•  Faster  decay  with  higher  levels  of  sub-task  interaction  or  interdependence. 

5.1.9  Potential  for  correcting  errors  and  re-deploying  resources 

The  ability  of  teams  to  self-correct  has  been  cited  as  a  critical  component  for  better  team  performance 
(e.g.  McIntyre  and  Salas,  1995).  Thus  the  potential  within  collectives  to  detect  and  compensate  for  errors 
is  likely  to  have  a  positive  influence  on  retention  of  collective  performance.  There  are  two  aspects  to 
this:  the  ability  of  individuals  in  the  collective  to  detect  and  self-correct,  and  the  ability  of  the  collective 
to  monitor  resource  use  and  re-deploy  teams  or  individuals  as  required.  This  capability  is  likely  to  relate 
not  only  to  feedback  and  cueing  (discussed  above)  at  an  individual  level  but  also  to  the  structure  of  the 
collective,  information  exchange,  and  the  interchangeability  of  sub-tasks  and  skills  within  the  collective. 

Collective  tasks  vary  in  the  extent  to  which  they  allow  for  correction  and/or  compensating  if  errors 
occur  during  task  performance.  Some  kinds  of  collective  tasks  are  performed  with  sub-units  working 
fairly  independently  on  different  sub-tasks.  For  example,  sub-units  preparing  a  defensive  position,  or 
multiple  but  similar  equipment  items  (e.g.  vehicles)  for  a  mission,  may  complete  their  tasks  at 
different  times,  and  be  re-deployed  to  assist  sub-units  falling  behind.  This  assumes  that  one  sub-unit 
will  know  that  another  needs  assistance,  has  the  knowledge  and  skills  to  provide  it,  and  is  directed  to 
do  so. 

Some  collective  tasks  may  allow  little  room  for  correction  if  one  unit  falls  behind  or  fails  to  perform 
their  part,  even  if  feedback  is  available  that  this  is  happening.  This  is  more  likely  to  occur  when  the 
tasks  require  specialized  skill  and  knowledge  or  one  sub-unit  in  the  collective  is  not  accessible.  For 
example,  some  sub-tasks  performed  by  tank  crew  members  are  not  amenable  to  correction,  even 
though  other  members  of  the  crew  are  aware  of  that  the  sub-task  is  falling  behind,  or  below  standard. 
Other  crew  members  may  not  have  the  knowledge  or  skills  required.  The  position  may  not  be 
accessible  from  another  part  of  the  tank.  The  crew  member  may  not  have  any  spare  capacity. 

Thus,  where  errors  or  delays  resulting  from  decay  in  the  skills  of  an  individual  member  or  sub-unit  in 
the  collective  can  be  detected  and  compensated,  and  retention  of  overall  collective  performance  may 
be  improved. 
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Predictions  for  collective  skill  retention: 

•  The  ability  to  detect  and  correct  errors  among  collective  members  will  improve  collective 
skill  retention. 

5.1.10  The  need  for  information  exchange  among  collective  members 

Communication  and  information  exchange  will  place  extra  demands  on  collective  members  during  a 
collective  task.  Information  exchange  may  be  needed  for  co-ordination  or  as  part  of  the  task  itself. 

The  amount  and  qualities  of  the  information,  and  the  paths  of  communication  among  members  of  the 
collective  will  affect  the  nature  of  the  communication  sub-tasks,  and  the  susceptibility  to  decay  of  the 
individual  tasks  as  well  as  the  effect  on  the  overall  task.  For  instance,  communication  among  many 
collective  members  are  likely  to  result  in  more  complex  decisions  and  procedures  with  higher  error 
rates  than  communication  that  occurs  among  only  a  few  collective  members. 

This  factor  will  interact  with  the  structure  of  the  collective.  Multiple  information  exchange  trajectories 
among  several  sub-units  organized  in  several  levels  imply  a  greater  need  for  effective  anticipation  of 
the  information  needs  of  other  collective  members  in  terms  of  timeliness  and  accuracy.  Information 
may  need  to  be  condensed  or  summarized  as  it  transits  through  the  collective.  This  implies  a  greater 
need  for  understanding  of  the  goals  and  tasks  of  other  collective  members,  and  for  awareness  in  terms 
of  feedback  and/or  cues  as  to  the  status  of  other  collective  members  tasks.  The  more  complex  the 
information  exchange  requirements  of  different  sub-tasks,  the  less  retention  that  can  be  expected  of 
individual  sub-tasks,  and  the  greater  the  impact  on  overall  retention  of  the  collective  task. 

With  more  organizational  levels,  relational  constraints  and  information  exchange  will  become  more 
complex  and  the  quality  of  information  exchange  is  more  prone  to  error  as  information  is  filtered  or 
compressed  as  it  moves  between  levels.  With  an  increase  in  either  the  number  of  vertical  or  horizontal 
levels,  the  negative  effect  of  poor  information  exchange  will  increase,  accelerating  the  impact  of  decay 
in  interactive  skills. 

Predictions  for  retention  of  collective  performance: 

•  Collective  tasks  with  more  complex  information  exchange  needs  in  terms  of  information 
quality,  quantity  and  number  of  people  involved  in  the  communication  will  be  less  likely 
to  be  retained  over  time. 

5.1.11  Cognitive  complexity  of  collective  sub-tasks. 

Examples  of  collectives  with  high  cognitive  demands  include  a  warship’s  operations  room  of  twenty 
to  thirty  people  dealing  with  multiple  and  simultaneous  threats  in  the  air,  on  the  surface  and 
underwater  and  the  command  team  of  an  army  combat  team  co-ordinating  the  activities  of  its  different 
units.  Among  these  teams,  the  cognitive  demands  of  different  sub-tasks  will  not  be  equal,  and  will 
depend  on  the  complexity  of  the  overall  collective  task. 

The  limited  research  available  suggests  that  individual  cognitive  or  decision-based  skills  such  as 
problem  solving  decay  in  the  region  of  20%  over  periods  of  3-6  months  (Sabol  and  Wisher,  2001)  with 
memory  for  more  complex  procedures  decaying  faster  than  less  complex. 

As  the  proportion  of  individual,  interactive  or  teamwork  sub-tasks  with  high  cognitive  or  information 
processing  demands  increases  within  the  collective,  the  more  closely  retention  of  the  overall  task  is  likely 
to  be  linked  to  cognitive  factors.  Examples  of  cognitive  tasks  include  knowledge  retrieval,  recall  of 
procedures,  and  decision  making  task  that  use  attention,  memory  and  information  processing  resources, 
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and  the  need  to  multi-task  using  different  styles  of  decision  making.  Considerations  include  the  degree  of 
planning  required  as  part  of  the  collective  task,  styles  of  decision  making  (for  example,  recognition 
primed  or  analytical),  the  degree  to  which  decision  making  is  or  can  be  made  procedural,  and  decision 
support  is  available. 

Predictions  for  retention  of  collective  performance: 

•  Less  retention  as  the  proportion  of  individual,  interactive,  or  team  work  tasks  requires 
recall  of  complex  procedures,  or  impose  higher  cognitive  demands  in  terms  of  attention, 
memory,  decision  making  or  multi-tasking. 

5.1.12  Summary  of  Collective  Task  Factors 

Several  features  of  collective  tasks  are  likely  to  affect  the  rate  at  which  performance  on  collective 
tasks  will  decay  over  time.  The  number  of  steps  in  the  sub-tasks,  the  number  of  distinct  specialized 
skills  required,  interdependence  among  critical  sub-tasks,  the  co-ordination  required,  and  the  amount, 
the  quality  and  pattern  of  information  exchange  and  cognitive  demands  are  all  likely  to  affect  retention 
of  performance  on  collective  tasks. 

A  theme  that  emerges  that  distinguishes  team  or  collective  tasks  from  individual  tasks:  co-ordination. 
Retention  is  likely  to  be  markedly  affected  by  the  degree  co-ordination  required  among  critical 
collective  sub-tasks  and,  as  well,  the  need  for  co-ordination  will  complicate  sub-tasks  that,  otherwise, 
could  be  treated  as  individual  tasks.  The  structure  of  collectives  and  collective  tasks  may  changed  by 
positions  and  sub-tasks  added  specifically  for  the  purpose  of  co-ordination.  Such  function  re-allocation 
is  likely  to  occur  when  the  sub-tasks  assigned  to  one  position  within  the  collective  exceed  the  capacity 
of  one  person  to  perform  them  acceptably. 

A  challenge  to  predicting  retention  is  the  interactive  effect  among  task  factors  and  other  aspects  of  a 
collective.  The  impact  of  collective  task  factors  is  likely  to  vary  depending  on  features  of  the 
collective  itself  such  as  its  structure  and  the  size  of  the  collective  sub-units  at  different  organizational 
levels  in  the  collective.  This  issue  is  addressed  in  the  next  section. 

5.2  Features  of  the  Collective 

Within  training  research  and  theory,  there  has  been  increasing  awareness  of  the  need  to  analyse  and 
understand  system-wide  influences  on  training  within  organizations  (Salas  and  Cannon-Bowers, 

2001).  At  the  level  of  small  team  training,  there  has  been  consideration  of  organizational  factors  that 
impact  on  team  skill  acquisition,  and  on  team  performance  (Cannon-Bowers  et  al.,  1995).  The 
purpose  of  this  section  is  incorporate  these  same  sort  of  considerations  into  understanding  collective 
skill  retention. 

The  section  covers  the  following  features  of  the  collective: 

•  Structure  of  the  collective  in  terms  of  sub-units  and  levels  of  organization. 

•  The  number  of  people  in  the  collective 

•  Leadership  structure 

•  The  potential  for  role  substitution 

•  Shared  mental  models  of  team  members  and  team  tasks. 

•  External  turnover  and  internal  turbulence  in  the  collective. 

•  The  social  environment  in  the  collective. 
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5.2.1  Structure  of  the  collective 

While  consideration  of  organizational  psychology  was  deemed  beyond  the  scope  of  this  review,  some 
aspects  can  be  considered,  and  consideration  in  greater  depth  should  undoubtedly  be  part  of  future 
research  into  the  issues  under  consideration. 

Hendrick  (1997)  offers  following  definition  of  an  organization:  “ The  planned  co-ordination  of  two  or 
more  people  who,  functioning  on  a  relatively  continuous  basis  and  through  division  of  labour  and  a 
hierarchy  of  authority,  seek  to  achieve  a  common  goal”.  Overlap  between  this  definition  with  the 
concept  of  a  team  and  of  a  collective  are  plain.  Related  topics  such  as  organizational  design, 
organizational  structure,  organizational  effectiveness,  and  many  others  are  likely  to  reveal  insights  into 
issues  related  to  collective  performance,  its  retention  under  different  circumstances,  and  approaches  to 
the  study  of  organizational  issues.  The  discussion  that  follows  only  scratches  the  surface,  without  the 
benefit  of  a  detailed  search  in  this  literature. 

Collective  structure  can  vary  in  the  number  of  levels  within  the  collective,  the  number  of  sub-units  at 
each  level  and  the  way  in  which  responsibility  is  divided  and  communication  channelled  within  this 
structure.  The  structure  will  influence  the  quality,  quantity  and  pattern  of  reporting  and  information 
exchange  among  elements  in  the  collective.  The  architecture  of  the  collective  will  also  define 
allocation  of  functions  to  individual  team  members  and  sub-units,  and  hence  the  degree  of 
specialisation  among  them  (Paris  et  al.,  2000).  The  more  complex  the  structure  the  more  knowledge 
required  to  construct  an  effective  mental  model  of  the  relationships  among  the  elements,  and  the  more 
difficult  retention  will  be.  In  turn,  this  is  likely  affect  some  of  the  task  factors  discussed  in  the 
previous  section,  such  as  co-ordination  and  re-deployment  of  resources,  timely  and  appropriate 
exchange  of  information,  and  others. 

Depending  on  the  needs  for  information  exchange  and  co-ordination,  more  highly  differentiated 
collectives  may  provide  more  communication  barriers,  necessitating  more  complex  communication 
procedures.  This  increases  the  probability  that  key  information  will  either  fall  between  the  cracks  or  be 
in  error,  affecting  co-ordination  of  the  collective  task. 

Using  collective  structure  to  predict  collective  skill  retention  is  prominent  in  a  model  created  by  Roth 
(1992).  Roth  explored  several  different  forms  of  military  units  and,  in  part,  used  the  structure  of  these 
units  to  predict  how  the  collective  skills  might  decay.  In  the  Roth  model,  the  related  concept  of 
“hierarchical  leadership”  was  given  the  most  dominant  influence.  Roth  argued  that  the  relative 
proportions  of  senior  leaders,  junior  leaders  and  non-leaders  were  important  predictors  of  collective 
skill  retention.  While  this  may  be  true,  it  is  likely  only  a  small  part  of  the  story  with  respect  to  the 
influence  of  collective  structure  on  retention  of  collective  performance. 

Predictions  for  retention  of  collective  performance: 

•  Collective  performance  will  decay  more  quickly  in  collectives  with  a  more  complex 
organizational  structure  in  terms  of  sub-units  and  levels  of  authority. 

5.2.2  Size  of  the  Collective 

As  the  number  of  people  in  collective  increases,  the  decay  of  a  collective  skill  is  also  likely  to  increase 
(Roth,  1992)  for  two  reasons.  First,  collective  skill  retention  may  decrease  as  a  simple  consequence  of 
individual  decay  functions  aggregated  across  multiple  people.  Secondly,  with  increasing  numbers 
comes  an  increased  demand  for  communication  and  co-ordination  among  members  and  sub-units  of 
the  collective.  Greater  complexity  is  associated  with  faster  decay,  other  things  being  equal.  Size  is 
also  likely  to  be  related  to  organizational  complexity. 
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On  the  other  hand,  as  argued  above,  depending  on  the  structure  of  the  collective  and  the  distribution  of 
skills,  sub-tasks  and  communication  channels,  and  the  level  of  redundancy  in  membership,  increased 
size  may  be  associated  with  increased  resilience  of  collective  memory  to  counter  instances  of  decay  in 
individual  skills,  an  increased  capability  to  compensate  for  errors,  and  better  redistribution  of  spare 
resources  to  adapt  to  change.  This  would  argue  for  an  increase  in  retention  of  overall  performance 
associated  with  increases  in  collective  size. 

There  may  also  be  other  relationships  between  collective  size  and  retention  of  collective  performance. 
In  very  large  groups,  for  example,  the  phenomenon  of  “social  loafing”  is  common  (Karau  and 
Williams,  1995).  Social  loafing  occurs  when  individuals  fail  to  put  in  their  best  effort,  presumably  in 
the  belief  that  other  members  of  the  group  will  “carry  the  load”.  Within  very  large  collectives,  a 
diminished  sense  of  personal  accountability  and  responsibility  may  impact  on  the  retention  of  complex 
skills.  In  large  collectives,  lower  levels  of  personal  accountability  and  responsibility  may  negatively 
influence  the  desire  or  ability  to  maintain  relevant  skills,  causing  collective  performance  to  erode  more 
rapidly. 

Another  factor  related  to  size  will  be  internal  personnel  turbulence  (e.g.  promotion,  or  re-assignment 
inside  the  collective)  and  external  personnel  turnover  (departure  from  the  collective  service  for  any 
reason).  The  smaller  the  team  or  group,  and  the  longer  its  prior  membership,  the  more  disruptive 
turbulence  and  turnover  is  likely  to  be  in  terms  of  retention  of  collective  performance.  This  issue  is 
discussed  in  a  later  section. 

In  the  end,  overall  collective  size,  per  se,  may  matter  less  than  the  interacting  factors  within  the 
collective,  for  instance,  the  size  and  structure  of  its  sub-groups  and  the  consistency  of  the  motivational 
environment  within  and  between  those  groups.  Nevertheless,  with  other  things  held  equal,  it  seems 
most  likely  that  an  increase  in  collective  size  will  reduce  retention  of  performance  in  overall  collective 
tasks. 

Predictions  for  retention  of  collective  performance: 

•  Increases  in  collective  size  will  reduce  retention. 

5.2.3  Leadership  structure 

Leadership  issues  may  be  regarded  in  terms  of  quantity  and  quality. 

In  quantitative  terms,  Roth  (1992)  discusses  the  number  of  leader  positions  in  a  collective,  as  related  to 
the  number  of  sub-units  and  teams.  Leadership  positions  are  regarded  as  focal  points  of  greater 
knowledge  of  collective  sub-tasks  and  interactive  skill  sets  in  the  collective,  the  loss  of  which  would 
be  particularly  detrimental  to  the  performance  of  collective  tasks.  The  more  leadership  positions  there 
are,  Roth  argues,  the  less  impact  there  will  be  as  a  result  of  the  loss  of  any  one. 

On  the  other  hand,  more  leadership  positions  will  be  correlated  with  a  greater  number  of  sub-groups 
within  the  organizational  structure.  Earlier,  greater  organizational  complexity  was  argued  likely  to 
inhibit  retention  of  collective  performance.  This  suggests  that  there  may  be  an  optimal  number  of 
leaders  in  relation  to  other  features  of  collective  tasks  and  structure,  above  and  below  which  collective 
performance  may  be  negatively  affected. 

In  qualitative  terms,  leaders  are  likely  to  know  more  about  their  subordinates  tasks  and  how  they 
interact,  and  possess  the  interactive  skills  (and  be  tasked)  to  detect  and  overcome  any  decay  in  their 
subordinates  skills  by  prompting  them  or  compensating  for  the  deficiency  in  some  other  way,  for 
example  re-deploying  resources  directly  within  that  team,  or  communicating  the  need  to  others  in  the 
collective. 
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Leaders  are  also  likely  to  possess  interactive  skills  necessary  for  co-ordinating  and  otherwise  facilitating 
performance  on  collective  tasks.  For  example,  creating  an  atmosphere  of  excellence,  for  example,  may 
increase  the  motivation  to  maintain  skills  at  an  individual  as  well  as  collective  level.  Decay  in  such 
interactive  skills  among  those  in  leadership  positions  can  be  expected  to  have  a  disproportional  effect  on 
collective  performance.  For  similar  reasons,  turnover  among  those  in  leadership  positions  can  be 
expected  to  have  a  greater  impact  on  retention  than  turnover  among  other  positions. 

Predictions  for  collective  skill  retention: 

•  Decay  in  interactive  skills  among  those  in  leadership  positions  will  have  a  greater  negative 
effect  on  collective  performance. 

•  Too  many  or  too  few  leadership  positions  will  negatively  affect  retention  of  collective 
performance. 

5.2.4  Role  substitution 

The  term  “ro/e  substitution ”  refers  to  the  potential  ability  of  members  of  a  collective  to  swap  sub-tasks 
and  still  perform  them  effectively.  This  concept  seems  closely  related  to  the  idea  of  interchangeability 
of  collective  sub-tasks.  Effective  swapping  assumes  common  knowledge  and  skill  in  relation  to  both 
the  collective  membership  and  the  requirements  of  collective  sub-tasks.  Roth  (1992)  uses  the  term 
“ position  redundancy ”  and  considers  the  potential  for  such  role  swapping  an  important  predictor  of 
improved  collective  skill  retention.  Dieterly  (1988)  distinguishes  “ task  redundancy ”  (the  ability  of 
more  than  one  team  member  to  perform  a  sub-task)  from  “ individual  redundancy ”  (added  team 
members  act  as  back-up  for  high  risk  critical  tasks).  Dieterly  associates  task  redundancy  with 
increased  system  reliability.  Dieterly  also  discusses  related  task  dimensions  such  as  “ position 
uniqueness”  (where  a  task  can  only  be  accomplished  by  one  position  in  the  team)  and  “ task 
uniqueness”  (a  sub-task  that  must  be  completed  for  completion  of  overall  collective  task).  Taken 
together,  Dieterly  uses  these  concepts  to  formulate  predictions  of  team  performance. 

When  the  potential  for  role  substitution  is  high,  individual  skill  decay  is  likely  to  impact  less  on  the 
performance  of  the  collective  as  a  whole  since  other  members  of  the  collective  will  be  likely  to  have 
retained  the  skills  and  knowledge  necessary  to  prompt  others  or  to  perform  the  task  themselves,  or  to 
identify  degradation  of  the  relevant  skill  and  the  need  to  do  something  about  it. 

Within  collectives,  the  potential  for  role  substitution  will  vary  according  to  the  level  of  specialisation 
of  its  members,  overlap  in  the  skill  requirements  of  sub-tasks,  and  sub-task  and  personnel  redundancy 
in  the  collective.  Where  collective  members  have  more  specialized  skills  and/or  the  collective  sub¬ 
tasks  require  specialized  skills  (two  things  that  seems  likely  to  go  together),  substituting  will  be  more 
difficult.  If  there  is  no  redundancy  in  terms  of  tasks  or  personnel,  re-deployment  to  achieve  role 
substitution  may  leave  functional  holes  elsewhere  in  the  organization. 

At  the  level  of  the  tank  crew,  for  example,  the  role  of  driver  is  very  distinct  from  the  role  of  gunner 
(Keesling,  1 995)  and  it  is  difficult  to  imagine  the  driver  substituting  for  the  gunner.  When  roles  are 
less  tightly  constrained  and  individual  skills  more  generic  across  the  collective  in  relation  to  the  task  to 
hand,  as  for  example  when  preparing  an  infantry  defensive  position,  swapping  may  be  easier.  In  such 
cases,  other  members  of  the  collective  can  pick  up  the  slack  caused  by  decay  in  sub-task  performance 
by  individuals,  and  maintain  overall  performance  of  the  collective. 

Thus,  the  likely  effect  of  role  substitution  will  interact  with  the  presence  of  generic  skills  among 
collective  members,  the  degree  to  which  the  overall  collective  task  permits  application  of  such  generic 
skills,  and  the  capability  of  the  collective  to  detect  a  problem  and  re-deploy  its  resources  appropriately. 
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Although  no  direct  support  was  found  for  the  assertion  that  role  substitution  will  enhance  the  retention 
of  collective  skills,  indirect  evidence  suggests  that  cross  training  enhances  team  performance.  Cross¬ 
training,  for  example,  requires  team  members  to  temporarily  swap  roles  with  other  team  members.  The 
goal  of  cross-training  is  to  provide  all  team  members  with  an  understanding  of  the  entire  team  function 
and  how  one’s  particular  tasks  and  responsibilities  interrelate  with  those  of  other  team  members.  This 
information  about  other  team  members  includes  appropriate  task  behaviours,  as  well  as  the  roles  and  the 
responses  required  by  the  team  mates  in  varying  situations,  and  allows  for  accurate  prediction, 
anticipation  of  information  exchanges  and  co-ordination  with  other  team  mates.  The  more  cross 
training  conducted  by  members  of  a  team  or  collective,  the  greater  the  assumed  sharing  of  team  mental 
models. 

Volpe  et  al.  (1996)  examined  cross-training  among  two  person  crews  for  a  simulated  air  combat  task. 
Without  cross  training,  participants  received  conventional  individual  training  relevant  only  to  the 
specific  function  that  they  needed  to  perform.  With  cross  training,  participants  learned  both  about 
their  own  specific  responsibilities,  and  about  those  of  the  other  team  members.  Teams  that  were  cross- 
trained  performed  better  at  their  own  task,  communicated  more  effectively,  and  were  better  able  to 
work  as  a  team  or  collective.  McCann  et  al.  (2000)  examined  cross-training  in  three  person  teams 
conducting  naval  surveillance  and  contact  identification  tasks  and  found  that  cross  trained  teams  did 
not  show  a  decrement  in  speed  or  accuracy  of  performance  as  a  result  of  unexpected  reconfiguration 
(role  substitution)  within  a  team,  while  teams  without  cross  training  did. 

While  this  work  relates  only  to  very  small  collectives  or  teams  and  does  not  speak  directly  to  the  issue 
of  retention  of  collective  performance,  it  does  suggest  that  a  higher  potential  for  role  substitution  may 
be  related  to  less  decay  in  collective  performance. 

Predictions  for  collective  skill  retention: 

•  Increased  potential  for  role  substitution  will  improve  collective  skill  retention. 

5.2.5  Team  Mental  Models 

Mental  models  are  typically  described  as  organized  knowledge  structures  that  allow  individuals  to 
predict  and  explain  the  world  around  them,  to  recognize  and  remember  relationships,  and  to  construct 
predictions  of  what  is  likely  to  occur  (Mathieu  et  al.,  2000).  Research  has  begun  to  address  the  extent 
to  which  individuals  construct  and  share  similar  mental  models  when  working  as  a  team  (Langan-Fox 
et  al.,  2000).  Effective  team  performance  is  held  to  require  multiple  shared  models  (Cannon  Bowers  et 
al.,  1993;  Mathieu  et  al.,  2000)  comprising  topics  focussed  around: 

•  Collective  tasks  and  sub-tasks: 

goals,  strategies,  procedures,  dependencies,  contingencies,  criticality. 

•  Team  interaction: 

role  responsibilities,  interdependencies,  information  channels. 

•  Team  membership'. 

Other  team  members  knowledge,  skills,  capabilities. 

•  Environmental  factors: 

Equipment,  terrain,  climate,  organizational  culture. 

In  short,  shared  mental  models  allow  team  members  to  be  “on  the  same  page”  in  pursuit  of  a  common 
goal.  Shared  mental  models  have  been  argued  to  improve  team  performance  (Langan-Fox  et  al., 

2000).  Mathieu  et  al.  (2000)  shows  that  a  higher  level  of  overlap  among  team  members’  mental 
models  is  associated  with  higher  levels  of  performance,  and  that  this  effect  is  mediated  by  team 
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processes  such  as  communication  and  co-ordination.  Cross  training  is  one  method  of  enhancing 
shared  team  mental  models. 

Although  no  research  was  found  to  directly  support  the  assertion  that  shared  mental  models  improve 
retention  of  collective  skills,  this  seems  likely.  Shared  mental  models  of  the  team  membership  should 
improve  anticipation  and  correction  of  decay  in  individual  skills.  Shared  mental  models  of  the  task 
should  also  enable  collective  members  to  substitute  for  each  other  more  effectively  and  co-ordination 
within  the  collective  should  be  more  resistant  the  decay. 

Decay  rates  of  different  knowledge  components  of  team  mental  models  are  likely  to  interact  with  other 
aspects  of  collective  structure,  collective  tasks  and  events  occurring  during  the  retention  interval.  For 
example,  increased  turnover  (people  leaving  the  collective  entirely)  will  affect  knowledge  about  team 
member  capabilities.  Increased  turbulence  (position  changes  within  the  collective)  may  be  the 
equivalent  of  cross-training  and  improve  tacit  knowledge  of  other  team  members’  tasks.  More  complex 
collective  structures  and  more  interdependent  tasks  will  increase  the  complexity  of  the  knowledge 
required  to  be  retained  for  the  mental  model,  and  increase  decay  rates.  Other  potential  interactions  need 
to  be  considered  in  detail. 

Predictions  for  retention  of  collective  performance: 

•  More  comprehensive  shared  mental  models  for  the  collective  and  the  collective  task  will 
improve  skill  retention. 

•  More  complex  mental  models  will  decay  more  rapidly. 

5.2.6  Turnover  and  Turbulence 

Two,  related,  sources  of  changes  in  collective  personnel  can  be  distinguished. 

•  turnover  (departure  from  the  collective  service  for  any  reason). 

•  turbulence  (e.g.  promotion,  or  re-assignment  inside  the  collective) 

High  rates  of  turnover  is  a  particular  problem  in  many  military  collectives.  Loss  of  technical 
specialists  with  better  job  prospects  outside  the  military  becomes  a  particular  challenge  as  dependence 
on  complex  technologies  increases. 

When  collective  members  leave,  they  take  with  them  knowledge  about  how  to  accomplish  a  collective 
task.  They  may  be  replaced  by  a  newcomer,  or  by  changes  made  within  the  collective.  The  newcomer 
may  bring  experience  or  capabilities  that  adds  value,  or  vice-versa.  Newcomers  may  reduce  collective 
task  performance  until  replacement(s)  learn  their  individual  and  interactive  tasks,  in  the  context  of  the 
current  collective  membership.  Even  in  the  best  case,  collective  task  performance  is  likely  to  diminish 
during  the  readjustment.  Roth  (1992)  believes  that,  in  general,  turnover  can  be  equivalent  to  an  entire 
month  without  practising  collective  skills,  although  the  basis  of  this  belief  is  not  clear. 

The  impact  of  turnover  on  collective  skill  retention  can  be  expected  to  interact  with  size  of  the 
collective,  the  role  of  the  collective  member,  and  the  skill  and  experience  of  the  substitute.  Within  a 
small  3  person  team,  for  example,  the  departure  of  one  team  member  on  collective  skill  retention  will 
likely  be  greater  than  would  turnover  of  one  private  within  an  infantry  company,  turnover  among 
leaders  could  be  expected  to  be  greater  than  non-leaders. 

A  closely  related  issue  in  the  literature  concerns  changes  in  the  roles  and  responsibilities  of  collective 
members,  or  collective  turbulence.  When  unit  members  change  the  positions  that  they  occupy  within  a 
team,  the  tasks  they  perform  are  also  likely  to  change.  Such  turbulence  frequently  occurs  in  response  to 
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staff  leaving  or  joining  the  collective,  and  for  promotions.  Team  members  must  then  readjust  learn  their 
new  roles  and  responsibilities  (Keesling,  1995). 

However,  if  turbulence  is  regarded  as  the  equivalent  of  cross  training,  then  a  positive  effect  can  be 
expected  with  respect  to  shared  mental  models  among  the  team  and  the  potential  for  role  substitution. 
For  example,  for  three  person  naval  surveillance  team  tasks,  McCann  et  al.  2000  compared  cross 
training  with  no  cross  training  within  a  standardized  training  period  with  two  results.  First,  level  of 
speed  and  accuracy  achieved  by  the  team  without  cross  training  exceeded  that  of  the  team  that 
received  cross  training.  This  result  was  explained  by  the  increased  training  time  per  position  received 
by  each  team  member  in  the  no  cross  training  condition.  However,  when  turbulence  occurred  (i.e. 
within  each  team,  positions  were  unexpectedly  swapped),  cross  trained  teams  showed  no  decrement  in 
speed  or  accuracy,  while  the  teams  without  cross  training  showed  an  appreciable  decrement.  An 
implication  of  this  is  that  if  the  cross  trained  teams  had  as  much  practice  in  each  position  as  teams 
without  cross  training,  turbulence  may  have  had  no  effect  on  performance,  and  final  performance 
levels  may  have  been  high. 

Thus,  while  there  may  be  a  period  of  deterioration  in  performance  shortly  after  a  period  of  turbulence, 
in  the  longer  term,  some  degree  of  cross  training  among  collective  members  may  actually  reduce  skill 
decay,  as  well  as  provide  other  benefits  described  in  the  section  on  shared  mental  models. 

The  long  term  effect  of  both  turnover  and  turbulence  on  retention  of  collective  skills  is  likely  to 
interact  with  levels  of  task  interdependence  and  the  potential  for  role  substitution  in  the  collective. 
Below  a  certain  point,  turnover  and/or  turbulence  may  have  promote  a  positive  cross  training  effect 
and  help  hold  decay  in  collective  performance  at  bay.  At  very  high  levels,  the  impact  of  turnover  may 
become  entirely  negative,  since  there  will  be  no  basis  for  role  substitution  or  shared  mental  models  - 
because  nearly  everyone  is  new. 

Predictions  for  collective  skill  retention: 

•  The  effect  of  low  rates  of  turnover  on  skill  retention  will  depend  upon  mental  models, 
individual  and  interactive  skills  of  the  replacements. 

•  The  effect  of  high  rates  of  turnover  will  be  to  reduce  collective  skill  retention. 

•  Turbulence  will  increase  shared  mental  models  and,  in  the  short  term,  reduce  collective 
performance  but,  in  the  long  term,  improve  collective  skill  retention. 

5.2.7  The  social  environment  within  the  collective 

Concepts  such  as  corporate  culture,  common  intent,  social  dynamics,  trust,  team  potency,  and  others 
are  concepts  that  are  supposedly  related  to  collective  performance,  and  potentially  retention  of 
collective  skills. 

A  positive  social  milieu  can  be  expected  to  impact  collective  skill  retention  in  different  ways. 

McCallum  et  al.  (1989;  cited  in  Swezey  and  Llaneras,  1997)  studied  13  teams  of  8-12  members  (and 
found  that  effective  teams  exhibit  behaviours  that  reinforce  effective  performance,  communicate  more, 
provide  feedback  and  help  where  performance  is  lacking,  and  exhibit  more  co-operative  behaviours. 
Dieterley  (1997)  cites  a  taxonomy  of  team  functions  by  Shifflet  et  al.  (1982)  that  includes  motivational 
functions  such  as  development  and  generating  acceptance  of  team  performance  norms,  establishing 
team  level  performance  rewards,  reinforcement  of  task  orientation  among  team  members,  balancing 
team  orientation  with  individual  competition  and  resolution  of  conflict.  Others  (e.g.  Amazon  and 
Sapienza,  1977;  Knight  et  al.,  1999)  distinguish  effective  management  teams  in  terms  of  their  ability 
to  both  resolve  and  make  use  of  affective  and  cognitive  conflict. 
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Such  aspects  of  the  social  environment  are  likely  to  result  in  higher  initial  levels  of  individual  and 
interactive  skills,  generation  of  more  comprehensive  and  reality  based  shared  team  mental  models,  and 
an  increased  potential  for  role  substitution.  Since  higher  initial  performance  levels  are  likely 
associated  with  longer  skill  retention  in  collectives  as  well  as  individuals,  it  is  a  short  step  to  conclude 
that  teams  or  collectives  with  positive  social  attributes  such  as  the  above  will  exhibit  better  retention  of 
collective  performance. 

Predictions  for  collective  skill  retention: 

•  The  presence  of  positive  social  attributes  within  collectives  is  likely  to  have  a  strong 
positive  effect  on  skill  retention. 

5.2.8  Summary  of  Features  of  the  Collective 

In  some  instances,  features  of  the  collective  are  difficult  to  distinguish  from  features  of  the  task  or  the 
characteristics  of  individuals.  As  size  of  the  collective  increases,  then  its  structure  in  terms  of  sub¬ 
groups  and  their  specialisation  becomes  more  closely  linked  to  the  nature  of  the  collective  task.  The 
structure  of  collectives  will  be  related  to  the  number,  type  and  diversity  of  tasks  it  is  expected  to 
perform.  Are  shared  team  mental  models  best  considered  attributes  of  the  individual  or  the  collective? 

Is  leadership  best  described  in  structural  terms  of  the  collective  (the  number  of  sub-groups  and  the  ideal 
span  of  control  /  supervision  leaders),  or  in  functional  terms  in  relation  to  the  needs  of  the  collective  task 
with  respect  for  particular  levels  of  effort  in  different  areas  of  specialisation,  or  in  qualitative  terms  in 
terms  of  the  attributes  required  by  those  in  leadership  positions. 

A  further  complication  is  that  many  features  are  interdependent  and/or  likely  to  have  inflection  points, 
above  and  below  which  the  positive  or  negative  nature  of  the  effect  may  change.  Teasing  out  the  relative 
effect  of  different  factors  in  relation  to  others  is  likely  a  Gordian  knot,  susceptible  only  to  broad  and 
relative  crude  solutions  at  the  current  state  of  knowledge. 

5.3  Characteristics  of  individuals  forming  the  collective 

Individual  differences  are  seen  by  some  as  an  impossibly  complex  factor.  Developers  of  the  UDA 
model,  for  example,  eliminated  consideration  of  individual  factors  due  to  practical  problems  of 
measurement  (Rose  et  al.,  1985a).  We  believe  the  role  of  individual  differences  deserves  to  be 
revisited  at  both  the  individual  and  the  collective  level. 

The  section  covers  characteristics  of  individuals  forming  the  collective,  as  follows: 

•  Individual  Aptitudes 

•  Diversity  of  experience 

•  Experience  in  collective  settings. 

5.3.1  Individual  Aptitude 

Individual  aptitude  is  frequently  cited  in  the  literature  as  a  predictor  of  individual  skill  acquisition  and 
retention  (e.g.  Hagman  and  Rose,  1983)  but  also  as  a  major  difficulty  in  deriving  general  laws  of 
acquisition  (Powers,  1976;  cited  in  Swezey  and  Llaneras,  1997).  For  skill  retention,  the  aptitude  of 
interest  is  typically  identified  as  aptitude  for  learning. 

Although  different  individual  aptitudes  are  consistently  related  to  different  acquisition  curves  under  a 
single  method  of  training,  these  curves  can  become  more  similar  when  training  is  customized  to 
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individuals  (Cronbach  and  Snow,  1976;  cited  in  Swezey  and  Llaneras,  1997).  High  ability  students, 
regardless  of  method  of  instruction,  typically  have  higher  initial  performance  levels  and,  consequently, 
slower  rates  of  improvement  (and  perhaps  slower  rates  of  decay).  The  result  is  often  instability  in 
acquisition  data,  especially  in  the  early  stages  of  complex  skill  acquisition,  likely  attributable  to  large 
within-subject  variations  in  aptitude  and  initial  performance  levels,  differences  in  rates  of  skill 
acquisition,  and  differences  in  learning  rates  across  the  various  stages  of  learning.  There  is  no  reason 
to  suppose  that  skill  retention  is  any  less  complicated  and  difficult  to  unravel,  especially  when  multiple 
individuals  are  concerned  at  the  level  of  the  collective. 

Not  surprisingly,  therefore,  the  relationship  between  aptitude  and  skill  retention  is  unclear  (Sabol  and 
Wisher,  2001).  Some  evidence  suggests  that  aptitude  on  its  own  does  not  predict  individual  skill 
decay,  as  the  rate  at  which  high  and  low  aptitude  individuals  show  skill  loss  has  been  shown  to  be 
fairly  consistent  (Hagman  and  Rose,  1983).  Other  evidence  suggests  that  high  aptitude  individuals  can 
achieve  over- learning  easier  than  can  low  aptitude  individuals  (Sabol  and  Wisher,  2001)  and  over¬ 
learning  is  frequently  cited  to  promote  better  individual  skill  retention  (e.g.  Hagman  and  Rose,  1983). 
This  would  suggest  that  higher  aptitudes  would  have  be  associated  with  better  retention.  Although  it 
not  been  tested  within  a  collective  context,  Roth  (1992)  suggests  that  individual  aptitude  should  be 
considered  in  future  collective  models.  Probably  future  work  should  consider  both  average  and 
individual  levels  of  aptitude  within  a  collective  unit,  especially  for  interactive  skills  critical  to 
performance  of  collective  tasks  with  interdependent  sub-tasks  (likely  most  collective  tasks)  and  for 
complex  specialized  individual  skills  required  for  sub-tasks  shown  to  be  on  the  critical  path  for  any 
given  collective  tasks. 

Individual  interactive  skills  such  as  co-ordination  and  communication  deemed  important  for  collective 
performance  and  retention,  may,  to  a  some  degree,  be  equated  to  leadership  skills.  Leadership  skills 
have  always  been  seen  as  important  in  the  context  of  military  collectives  but  to  a  large  extent  are 
believed  to  be  acquired  by  tacit  on  the  job  learning  (Horvath  et  al.,  1994a,  1994b).  If  this  is  true,  then 
learning  aptitudes  will  be  particularly  critical  for  acquisition  tacit  interactive  skills  and,  hence, 
influential  in  determining  collective  skill  retention. 

Classifying  tasks  into  taxonomies  according  to  underlying  skill  requirements  is  a  common  approach 
for  many  issues  in  training.  For  example,  Driskell  et  al.,  (1992;  cited  in  Swezey  and  Llaneras,  1997) 
have  distinguished  between  physical  and  cognitive/knowledge  task  components,  with  different 
acquisition  and  retention  characteristics.  Different  classification  approaches  include  those  which 
organize  task  requirements,  and  those  which  organize  presumed  learning  processes  underlying  task 
performance,  often  based  on  task  analytic  methods  (Gael,  1998).  For  instance,  Dieterly  (1988) 
outlines  Schifflet  et  al.’s  (1982)  taxonomy  of  team  functions  as  follows. 
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Function 

Item 

Orientation 

1.  Information  exchange  regarding  members  and  resources 

2.  Information  exchange  regarding  team  task  and  goals  /  missions 

3.  Information  exchange  regarding  environmental  characteristics  and  constraints 

4.  Priority  assignment  among  tasks. 

Resource 

distribution 

1.  Matching  member  resources  to  task  requirements 

2.  Load  balancing 

Timing  or 
pacing 

1.  General  activity  pacing 

2.  Individually  oriented  activity  pacing 

Response  co¬ 
ordination 

1.  Response  sequencing 

2.  Time  and  position  co-ordination  of  responses 

Motivation 

Development  of  team  performance  norms. 

Generating  acceptance  of  team  performance  norms. 

Establishing  team-level  performance-reward  linkages 

Reinforcement  of  task  orientation 

Balancing  team  orientation  with  individual  competition 

Resolution  of  performance-relevant  conflicts. 

Table  3:  Taxonomy  of  Team  Functions  (from  Shiflett  et  al.,  1982) 

It  is  likely  that  taxonomies  or  aptitude  profiles  for  interactive  team  work  skills  already  exist.  If  so,  it 
would  be  worthwhile  to  investigate  the  relationship  such  profiles  with  the  speed,  acquisition  and 
retention  of  interactive  and  teamwork  skills,  and  acquisition  and  retention  of  performance  on  selected 
collective  tasks. 

Thus,  for  collective  tasks,  indicators  of  aptitude  should  be  broadened  to  include  more  than  just 
aptitude  for  learning  of  individual  sub-tasks.  Individual  differences  in  interactive  skills  such  as  the 
ability  to  co-ordinate  and  communicate  with  others  are  also  likely  to  predict  the  long  term  retention  of 
collective  skills.  Such  a  distinction  is  prominent  in  the  team  work  literature’s  description  of  task  work 
and  teamwork  (McIntyre  and  Salas,  1995). 

Predictions  for  retention  of  collective  performance: 

•  Higher  aptitude  for  learning  tacit  interactive  skills  and  elements  of  team  mental  models 
will  be  related  to  better  retention  of  performance  on  collective  tasks. 

•  Higher  aptitude  for  individual  specialist  skills  on  the  critical  path  for  collective  tasks  will 
be  related  to  better  retention  of  performance  on  collective  tasks. 

5.3.2  Diversity  of  Expertise 

Diversity  of  expertise  in  a  collective  will  be  related  to  level  of  specialisation  required  by  different  sub¬ 
tasks  in  the  collective  task,  and  the  organization  of  sub-groups  within  the  collective  structure.  At  the 
individual  level,  greater  diversity  of  specialisation  with  small  groups,  at  whatever  level  in  the 
organization  but  most  studied  in  relation  to  management  teams  or  product  or  software  development 
teams,  has  been  related  to  greater  levels  of  cognitive  and  affective  conflict  (  Amazon  and  Sapienza, 
1977),  and  as  providing  obstacles  to  information  exchange  and  fusion  in  pursuit  of  solutions  to 
collective  problems  requiring  integration  of  diverse  expertise.  These  difficulties  arise  often  because  of 
barriers  to  comprehension  across  different  knowledge  domains,  or  real  or  perceived  differences  in  the 
motivation  or  competence  of  other  individuals  or  sub-groups  (e.g.  Knouse,  1996;  Stenger,  1997). 
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Oversimplifying,  the  key  to  overcoming  such  challenges  in  pursuit  of  collective  goals  is  often  cited  as 
possession  by  group  leaders  of  superior  interactive  or  facilitation  skills  (  Amazon  and  Sapienza,  1977). 
The  probable  importance  of  such  skills  to  collective  performance  has  been  already  noted  several  times. 

Another  likely  influence  is  the  duration  of  the  group  -  those  groups  with  longer  durations  are  more 
likely  to  expend  the  effort  needed  to  learn  to  work  together,  as  well  as  having  more  opportunity  for 
tacit  learning  about  capabilities  of  other  team  members.  Especially  at  the  level  of  larger  collectives,  or 
multi-arm,  joint  service,  or  multi-national  collectives,  medium  size  collectives  may  be  convened  at 
relatively  short  notice  with  little  experience  one  of  the  other  to  perform  collective  tasks  for  which  they 
have  been  trained  only  in  isolation  one  from  the  other.  (Historically  this  has  been  so,  and  current  or 
recent  operations  in  the  Balkans  and  in  Afghanistan  would  also  seem  to  support  this).  In  fluid  long 
term  operations,  collective  tasks  at,  say  the  battle  group,  brigade  or  naval  task  force  level,  may  be 
formed  from  diverse  and  specialized  sub-units  with  little  direct  experience  one  of  the  other. 

Collectives  with  great  diversity  in  expertise  among  their  members  or  sub-groups  are  likely  to  have 
more  difficulty  maintaining  a  common  view  of  the  task,  may  be  less  likely  to  maintain  skills  outside 
their  own  area  of  interest,  and  find  communication,  information  exchange  and  task  co-ordination, 
particularly  for  sub  tasks  that  resist  proceduralization,  more  challenging.  This  would  suggest  that 
higher  levels  of  diversity  in  expertise  would  predict  lower  levels  of  collective  skill  retention. 

Predictions  for  retention  of  collective  performance: 

•  Greater  diversity  of  expertise  within  the  collective  will  be  related  to  less  retention  in 
collective  skills. 

5.3.3  Experience  in  Collective  Settings 

The  experience  of  collective  members  may  be  thought  of  in  different  ways:  prior  to  joining  the 
collective;  prior  to  training  on  the  collective  task(s);  during  the  retention  interval.  Also,  there  are 
different  ways  in  which  such  experience  may  relate  to  retention  of  performance  on  collective  tasks. 

For  example,  through  the  positive  or  negative  transfer  of  training  from  past  experiences  to  present 
performance  or  by  adding  to  the  repertoire  of  knowledge  and  skills  available  within  the  collective, 
thereby  enhancing  the  potential  within  the  collective  for  error  recovery  or  role  substitution.  The 
degree  of  similarity  of  shared  experiences  among  collective  members  may  also  contribute  to  sharing  of 
mental  models  among  collective  members  to  a  greater  or  lesser  degree.  The  degree  of  similarity  might 
range  from  being  in  the  same  collective  performing  the  same  collective  task  performing  the  same  sub¬ 
task  under  similar  conditions,  to  being  in  a  similar  collective  but  with  different  people,  performing  a 
different  collective  task  or  a  different  sub-task,  under  dissimilar  conditions.  The  permutations  and 
combinations  would  be  many,  some  aspects  may  be  key.  One  likely  one,  often  cited  anecdotally,  is 
experience  of  actual  combat. 

Pigeau  and  McCann  (1995,  2000)  assert  that  common  intent  (believed  especially  important  for 
achieving  collective  goals  as  represented  by  command  intent  when  planning  and  procedures  fall  apart 
under  the  fog  of  war)  is  achieved  among  members  of  a  collective  largely  as  a  result  of  shared 
experiences  with  other  members  of  the  collective  as  well  as  learning  held  in  common,  perhaps 
stretching  back  into  common  cultural  antecedents  to  military  experience.  The  basis  of  regimental 
tradition  claimed  to  underlie  excellence  in  military  performance  (Winslow,  1998)  is  generated,  to  a 
large  extent,  by  stressing  the  common  history  among  members  of  the  regiment  and  having  members  of 
the  regimental  “family”  stay  in  or  return  to  the  regimental  collective  whenever  practicable.  (Witness 
the  protests  whenever  downsizing  forces  formerly  separate  British  regiments  to  combine.) 
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There  are  many  reasons  to  suppose  the  cross  section  of  experience  among  collective  members  of 
collective  tasks  might  impact  on  skill  retention.  Greater  experience  performing  collective  tasks  may 
result  in  over-learning  of  some  components  of  interactive  skills,  making  these  skills  more  resistant  to 
forgetting.  Experience  working  (or  playing)  with  others,  provides  the  opportunity  to  develop  the 
knowledge  structures  needed  to  attend  to  the  cues  and  information  provided  by  other  team  members, 
either  as  generic  skills,  or  to  build  mental  models  of  particular  members  of  one’s  own  collective.  For 
example,  participation  in  team  sports  systematically  organized  to  build  networks  across  smaller  units 
in  the  wider  collective  will  help  build  different  aspects  of  team  mental  models,  and,  presumably, 
enhance  ability  to  predict  how  others  will  respond  and  lead  to  better  performance  on  future  collective 
tasks. 

There  is  theoretical  agreement  that  teams  who  work  together  for  longer  periods  of  time  will  show 
higher  levels  of  performance  (Mclntrye  and  Salas,  1 995)  but  we  are  not  aware  of  any  empirical 
evidence  that  collectives  in  which  members  have  broad  experience  working  together  (or  as  a  members 
of  other  collectives)  show  retain  collective  performance  better,  though  it  seems  likely.  However,  such 
concepts  are  likely  to  have  been  examined  in  the  literature  on  occupational  and  organizational 
psychology,  and  merits  more  thorough  investigation  than  has  been  possible  in  this  project. 

Predictions  for  collective  skill  retention: 

•  For  individual  skills,  the  effect  of  previous  experience  in  collective  settings  and  with 
collective  tasks  will  depend  on  the  transfer  of  training  between  those  experiences  and  the 
collective  task  of  interest. 

•  Shared  experiences  of  collective  settings  and  collective  tasks  among  collective  members 
will  contribute  to  the  shared  mental  models  of  team  members.  The  effect  on  collective 
performance  will  depend  on  the  degree  to  which  the  elements  of  the  shared  team  mental 
model  match  the  collective  membership,  context,  and  task  of  interest. 

5.3.4  Summary  of  Individual  Factors 

Particularly  among  leaders  of  sub-groups  within  collectives,  individual  differences  in  aptitude  for 
learning  tacit  interactive  skills  and  knowledge  components  of  team  mental  models  as  well  as  differences 
in  individual  skills  associated  with  performance  of  sub-tasks  on  the  critical  path  of  collective  tasks,  are 
likely  strong  contributors  to  both  acquisition  and  retention  of  performance  on  collective  tasks. 

Diversity  of  expertise  among  collective  members  will  likely  be  related  to  collective  structure,  in  terms 
of  sub-group  functions.  Co-ordination  of  the  efforts  of  very  diverse  speciality  sub-groups  and  pooling 
of  expert  knowledge  among  the  leaders  of  these  sub-groups  will  present  a  challenge  to  acquisition  and 
retention  of  collective  performance,  especially  in  situations  where  pooling  of  knowledge  is  required  for 
problem  solving,  or  when  situations  change  during  the  performance  of  the  collective  task.  Experience 
in  collective  settings  may  enhance  or  interfere  with  collective  performance,  depending  on  whether  there 
is  a  positive  or  negative  transfer  effect  between  the  experience(s)  in  question,  or  the  degree  the 
experience  is  held  in  common  and  contributes  to  the  shared  mental  model  of  the  different  elements  of 
collective  functioning. 

Overall,  individual  differences  among  collective  members  are  likely  to  play  a  prominent  role  in 
retention.  Despite  their  complexity,  relevant  aspects  of  occupational  and  organizational  merit  further 
investigation  with  respect  to  their  application  potential  for  prediction  of  collective  performance  and 
retention. 
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5.4  Features  of  the  training  received  by  the  collective 

Training  issues  relate  to  the  nature  and  focus  of  training  received  by  individual  collective  members  in 
individual  and  interactive  skills,  and  the  training  that  the  collective  receives  as  a  whole  in  the 
collective  task  of  interest  and  in  other  collective  tasks  (before  or  during  the  retention  period),  the 
effects  of  which  may  transfer  positively  or  negatively  to  overall  collective  performance  on  the  task  of 
interest. 

The  section  covers  the  following  features  of  training  received  by  the  collective: 

•  Level  of  learning  after  initial  training 

•  The  match  between  the  training  context  and  the  retention  context; 

•  Training  received  as  a  collective; 

•  Training  approaches  used  to  achieve  collective  performance. 

5.4.1  Level  of  learning  after  initial  training 

For  individual  skills,  the  degree  of  initial  learning  as  a  result  of  training  has  been  argued  to  be  one  of  the 
most  important  predictors  of  skill  retention  (Sabol  and  Wisher,  2001;  Spears,  1991a;  Bodilly  et  al.,  1986; 
Farr  1987).  The  greater  the  initial  learning,  the  slower  the  rate  of  forgetting.  This  effect  is  so  strong  that 
some  argue  that  anything  that  leads  to  higher  levels  of  initial  learning  will  strengthen  skill  retention 
(Swesey  and  Llaneras,  1997).  Differences  among  individuals  in  terms  of  original  learning  seem  more 
powerful  determinants  of  retention  than  differences  in  forgetting.  This  effect  has  held  in  a  wide  range  of 
contexts  including  motor,  procedural  and  verbal  learning  (Swezey  and  Llaneras,  1997)  and  armour  skills 
(Rowatt  and  Shlechter,  1993).  Retention  can  be  expected  to  improve  where  there  is  increased  practice 
beyond  mastery  to  induce  over-learning,  and  when  learners  are  required  to  overcome  instances  of 
contextual  interference  or  provided  with  opportunities  for  deeper  processing  that  results  in  better 
organization  of  material  for  later  retrieval.  This  leads  to  the  suggestion  that  evaluation  of  the  training 
methods  used  may  be  an  effective  method  of  predicting  retention  in  combination  with  measurement  of 
performance  on  completion  of  training. 

However,  there  is  increasing  emphasis  of  the  importance  of  distinguishing  between  acquisition  and 
learning,  and  identification  of  the  apparent  paradox  that  training  conditions  that  maximize  longer 
retention  and  greater  resistance  to  altered  contexts  may  not  result  in  result  in  maximum  performance 
during  acquisition  (e.g.  Schmidt  and  Bjork,  1992).  Thus  the  strategy  of  using  end  of  training  evaluations 
to  predict  performance  on  the  job  or  outside  the  training  environment  may  not  be  appropriate,  with 
implications  for  validation  of  approaches  to  predicting  collective  skill  retention.  This  raises  the  question 
of  just  how  should  skill  retention  be  measured.  This  methodological  issue  is  addressed  in  a  later  section. 

These  observations  are  likely  to  apply  to  the  learning  of  collective  tasks  in  two  ways.  First,  individual 
learning  of  skills  critical  to  the  collective  task  and  of  interactive  skills  (e.g.  skills  related  to  co¬ 
ordination)  are  likely  to  respond  in  a  similar  manner.  Secondly,  there  is  the  issue  of  performance  on 
the  overall  collective  task.  Presumably,  training  strategies  can  be  conceptualized  in  the  same  way  for 
collective  tasks  whereby  increased  practice  will  lead  to  over-learning  and  contextual  interference  will 
lead  to  better  internal  organization  of  mental  models  with  much  the  same  overall  effect  on  the 
collective  as  on  individuals,  provided  collective  membership  remain  the  same,  in  the  same  positions 
(i.e.  turnover  and  turbulence  are  low).  In  the  context  where  there  are  limited  training  resources,  the 
question  arises  whether  repeated  practice  or  contextual  interference  training  is  more  effectively 
focussed  on  critical  individual  skills,  individual  interactive  skills,  or  training  of  the  collective  task(s)  as 
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a  whole,  and  which  strategy  will  lead  to  better  retention  of  performance  on  collective  tasks,  under  field 
conditions.  This  appears  to  be  an  open  question. 

Predictions  for  retention  of  collective  performance: 

•  Performance  after  initial  training  of  individual  skills  related  to  critical  collective  sub-tasks  or 
to  interactive  skills  may  not  predict  long  term  retention  of  those  skills. 

•  Training  strategies  such  as  increased  practice  beyond  mastery,  contextual  interference,  or 
internal  elaboration  will  be  related  to  better  retention. 

5.4.2  Match  between  Training  Context  and  Retention  Context 

Transfer  of  training  is  closely  related  to  many  training  issues  and  factors  affecting  positive  or  negative 
transfer  of  training  influence  individual  skill  retention.  Transfer  of  training  refers  to  the  way  that 
previous  or  intervening  learning  affects  new  learning  or  subsequent  performance:  in  particular,  how 
well  initial  learning  transfers  to  new  situations.  The  effect  may  be  to  inhibit  performance  (negative 
transfer)  or  improve  it  (positive  transfer).  This  issue  lies  at  the  heart  of  the  measurement  of  retention: 
i.e.  testing  after  training  and  re-test  after  a  prescribed  interval  or  intervals.  Some  believe  that,  despite 
enormous  sums  of  money  spent  on  industrial  training,  possibly  as  little  as  1 0%  actually  transfers  to  on 
the  job  performance  (Swezey  and  Llaneras,  1977). 

Over-simplified,  transfer  of  training  is  thought  to  relate  to  the  degree  of  similarity  of  the  two  situations 
and  the  two  responses  required.  The  match  between  training  and  retention  is  directly  related  to  the 
number  of  cues  present  in  the  retention  context  that  assist  recall  of  information  learned  during  training. 
However,  most  complex  situations  resist  measurement  in  such  terms.  The  components  of  a  complex 
situation  can  be  considered,  like  collective  tasks,  to  comprise  sub-tasks,  among  any  of  which  there 
may  be  different  levels  of  transfer.  Swezey  and  Llaneras  (1977)  use  the  example  that,  overall,  positive 
transfer  between  every  component  of  one  aircraft  and  another  except  one  critical  one,  might  still  result 
in  an  unacceptable  deterioration  in  overall  performance.  Much  the  same  difficulties  apply  to  collective 
tasks. 

A  common  assumption  has  long  been  that  the  greater  the  match  or  fidelity  between  the  training 
simulator  and  the  operational  environment,  the  greater  the  likelihood  of  a  positive  transfer  of  training 
for  individual  skills.  The  match  between  training  and  retention  context  is  also  directly  related  to  the 
number  of  cues  present  in  the  retention  context  that  will  assist  in  the  recall  of  information  learned 
during  training. 

More  recently,  the  effectiveness  of  exact  physical  fidelity  has  been  questioned.  Interest  has  turned  to 
consider  the  importance  of  the  learning  support  provided  and  instructional  approaches.  In  some  cases 
it  is  argued  that  exact  physical  fidelity  limits  the  use  of  training  strategies  known  to  increase  transfer 
or  retention  such  as  corrective  feedback,  or  that  the  complexity  often  inherent  in  a  high  degree  of 
fidelity  distracts  and  confuses  learners.  This  trend  has  been  accompanied  by  a  distinction  between 
physical  and  psychological  fidelity:  the  degree  to  which  the  trainee  (or  trainees  in  the  case  of  a  team  or 
collective)  perceives  the  training  medium  as  duplicating  the  equipment  and  or  the  task  situation.  Some 
propose  that  qualitative  differences  in  the  degree  of  fidelity  according  to  the  stage  and  type  of  learning. 
Early  procedural  training  benefiting  from  low  fidelity,  initial  skill  acquisition  needing  greater  fidelity 
in  terms  of  equipment  and  the  context,  and  finally  skill  acquisition  needing  not  only  the  highest  level 
of  fidelity,  but  also  the  greatest  degree  of  contextual  variety  (Swezey  and  Llaneras,  1977). 

An  implication  for  skill  retention  may  be  that  the  test  -  retest  conditions  should  take  into  account  the 
stage  of  training  to  which  the  collective  task  has  been  taken.  For  larger  collectives,  it  is  likely  that 
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training  at  the  whole  collective  level  is  so  infrequent,  that  the  collective  as  a  whole  will  not  have 
progressed  beyond  an  intermediate  stage  of  learning,  and  that  skill  retention  should  be  considered  only 
as  a  springboard  from  which  to  launch  training  aimed  at  preparing  for  a  specific  mission.  This 
suggests  that  collective  skill  retention  models  should  take  progression  across  different  levels  of 
learning  in  relation  to  type  of  maintenance  training  rather  than  just  final  operational  learning  targets. 

For  collectives,  factors  affecting  transfer  of  training  are  likely  to  be  important  predictors  of  collective 
performance.  Because  of  the  size  of  most  military  collectives  much  training  done  as  a  collective  is 
done  in  the  field,  making  it  impossible  to  retest  in  exactly  the  same  environment  in  which  training 
occurred.  Flowever,  as  access  to  distributed  simulators  increases,  then  the  potential  for  controlling  the 
conditions  under  which  collective  tasks  are  trained  and  assessed  may  permit  the  impact  of  such  factors 
to  be  better  evaluated.  In  fact,  simulator  training  may,  for  some  high  risk  or  high  expense,  collective 
tasks,  represent  a  more  realistic  and  less  expensive  way  of  training  collectives.  An  example  might  be 
the  training  of  warship  operations  room  teams  to  respond  to  multiple  air  and  surface  threats  under 
realistic  engagement  conditions,  without  the  expense  of  sending  ships  to  sea  and  using  simulated 
rather  than  real  weapons  such  as  anti-ship  and  anti-aircraft  missiles.  (Note  also  the  earlier  comment 
about  the  potential  utility  of  navy  team  training  simulators.) 

One  approach  to  measuring  transfer  of  training  holds  some  interest  for  predicting  retention.  This  is  the 
simulator  evaluation  approach  proposed  by  Adams  (1979).  As  described  in  Swezey  and  Llaneras 
(1997),  this  appears  similar  to  a  verification  approach,  in  that  the  effectiveness  of  a  simulator  can  be 
predicted  by  the  degree  that  it  possesses  features  and  its  use  adheres  to  scientific  principles  of  training. 
Much  the  same  principle  might  be  advanced  for  predicting  retention  of  collective  performance:  i.e.  the 
degree  to  which  the  combined  training  of  critical  individual  skills,  interactive  skills  and  team  work  has 
adhered  to  appropriate  training  principles  known  to  promote  retention. 

As  in  the  case  of  individual  skill  retention,  the  interval  between  when  skills  are  trained  and  used  is  an 
influential  predictor  of  skill  retention.  Time  without  training  is  typically  argued  to  promote  the  decay  of 
skills  by  breaking  down  the  cognitive,  perceptual  or  motor  processes  which  underlie  the  performance  of 
skills,  or  by  making  it  more  difficult  for  these  processes  to  be  either  accessed  or  performed.  For 
collective  tasks,  we  would  expect  time  without  training  to  impact  not  only  on  levels  of  individual  skill, 
but  on  the  ability  to  perform  these  skills  as  a  member  of  a  interdependent  collective.  The  length  of 
retention  interval  can  be  expected  to  interact  with  type  of  experiences  undertaken  during  it  since  these 
may  strengthen  or  interfere  with  retention  to  the  degree  those  experiences  act  as  further  practice  or  not. 

There  been  very  little  research  on  the  impact  of  retention  interval  on  collective  skill  retention  (Roth, 
1992).  Work  on  a  limited  number  of  collectives  suggests  an  average  of  1 1%  decrease  in  performance 
after  one  month  without  practice  or  training  on  the  collective  tasks  identified,  but  clearly  more  work  is 
needed  to  expand  knowledge  in  this  area. 

Predictions  for  retention  of  collective  performance: 

•  Intervening  or  test  conditions  equating  to  negative  transfer  of  training  will  reduce 
retention. 

•  Intervening  or  test  conditions  equating  to  positive  transfer  of  training  will  improve 
retention. 

•  Longer  retention  intervals  will  reduce  retention. 
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5.4.3  The  training  received  as  a  collective  unit 

Two  aspects  of  training  as  a  collective  unit  can  be  considered:  training  on  the  target  collective  task, 
and  training  on  other  collective  tasks  before  and  during  the  retention  interval. 

While  training  on  the  target  collective  task  will  probably  have  the  greatest  benefit,  training  as  a 
collective,  in  whole  or  in  part,  on  other  collective  tasks  can  be  expected  to  have  some  transfer  effect 
depending  on  the  overlap  between  the  tasks.  Working  (or  playing)  together  with  other  collective 
members  on  other  tasks  can  be  expected  to  contribute  to  interactive  skills,  to  the  building  of  mental 
models  about  the  capabilities  of  other  collective  members  with  respect  to  critical  individual  and 
interactive  skills  that  contribute  to  given  collective  tasks  (Mathieu  et  al.,  2000). 

Some  individual  and  interactive  knowledge  skills  that  contribute  to  the  collective  task  can  be  learned, 
to  some  level,  in  isolation  from  other  collective  members,  or  by  working  on  other  collective  tasks. 
However,  training  together  as  a  collective  on  a  particular  collective  task  not  only  allows  permits 
further  practice  and  confirms  the  capability  of  the  collective  to  perform  the  task  to  the  required 
standard,  but  adds  a  team  work  dimension. 

When  collective  members  train  together,  they  practice  not  only  the  individual  steps  in  their  own  sub¬ 
tasks  and  the  steps  in  the  collective  task,  but  also  learn  to  work  together,  to  exchange  information 
effectively  and  co-ordinate  their  actions,  and  to  know  what  to  expect  from  their  team-mates.  This  will 
be  particularly  valuable  to  two  classes  of  collective  member:  those  performing  interdependent  sub¬ 
tasks  and  those  responsible  for  co-ordinating  within  and  between  sub-groups  within  the  collective. 
Time  critical  sub-tasks,  and  sub-tasks  involving  extensive  information  exchange  and  interaction 
among  collective  members  will  also  benefit  from  training  or  practice  as  a  collective. 

With  each  added  level  of  organizational  complexity,  the  task  of  sub-group  co-ordinators  can  be 
expected  to  change.  For  example,  when  a  basic  unit  such  as  a  tank  crew  operates  in  isolation,  the  crew 
commander  is  responsible  primarily  for  his/her  own  individual  sub-task,  and  for  co-ordinating  the  tasks 
of  the  other  crew  members  in  relation  the  goal  of  the  tank.  As  other  levels  of  organization  are  added  to 
the  collective  (other  tanks,  infantry,  engineers,  artillery,  fast  air),  the  co-ordination  and  tactical 
challenges  of  the  crew  commander  increase  and  multi-tasking  becomes  more  necessary,  leaving  the 
leader  with  less  cognitive  capacity  to  cope  with  the  task  at  hand  than  when  the  tank  crew  operated  in 
isolation.  Dealing  with  the  consequences  of  added  organizational  complexity  are  difficult  to  learn  in 
isolation,  and  tacit  learning  will  be  required. 

Thus,  as  the  number  of  levels  or  sub-teams  in  a  collective  increase,  the  benefit  of  training  together  will 
increase.  Individual  skills  on  the  critical  path  and  interactive  sub-tasks  can  be  practised  more 
realistically  as  a  collective.  Multi-tasking  strategies  required  for  co-ordination  across  several 
organizational  levels  can  also  be  learned.  Many  aspects  of  team  mental  models  can  be  refined,  or 
developed:  particularly  those  relating  to  the  performance  capabilities  of  other  collective  members. 

Tacit  learning  will  occur. 

However,  there  is  an  important  caveat:  the  degree  of  learning  that  occurs  is  likely  to  depend  on  the 
instructional  strategies  employed,  and  the  stage  of  learning  reached  by  individuals  or  sub-groups  with 
respect  to  knowledge  of  the  collective  task,  as  well  as  individual  and  interactive  skills.  Simply 
practising  together  as  a  collective  on  the  collective  task  may  not  always  have  any  learning  impact.  For 
instance,  it  is  likely  that  learning  as  a  collective  progresses  through  stages  much  as  individual  tasks.  If 
this  true,  in  terms  of  fidelity,  for  instance,  early  conditions  of  training  on  a  collective  task  should  not 
be  too  realistic,  or  some  or  all  collective  members  may  become  confused  as  a  result  of  too  much 
complexity,  and  learn  little  -  much  in  the  same  way  as  learning  of  individual  skills  can  be  inhibited  if 
too  much  fidelity  is  introduced  too  early.  Likewise,  practise  without  detailed  knowledge  of  results  and 
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corrective  guidance,  is  unlikely  to  result  in  learning  or,  worse,  may  result  in  learning  the  wrong 
lessons. 

Given  the  relative  infrequency  of  training  as  a  collective,  particularly  for  larger  collectives,  it  is 
possible  that  the  level  of  learning  achieved  for  a  given  collective  task  may  not  have  ever  progressed 
beyond  the  initial  associative  stage,  or  the  intermediary  procedural  knowledge  stage,  and  that  the 
autonomous  stage  dependent  on  frequent  extra  practice  will  not  have  been  reached.  For  managing 
training  strategies,  knowing  the  stage  to  which  learning  on  collective  tasks  or  sub-tasks  has  progressed 
is  critical  in  predicting  decay.  To  estimate  the  stage  of  learning  achieved,  an  estimate  transfer  of 
training  from  one  type  of  collective  activity  to  another,  with  respect  to  the  different  individual  skills 
and  knowledge  required  for  performance  of  the  collective  task,  may  be  more  meaningful  than  the 
number  of  times  a  particular  collective  task  has  been  practised. 

In  conclusion,  we  would  expect  that  a  collective  that  trained  as  a  unit  would  maintain  their 
performance  as  a  collective  better  over  time  than  collectives  that  did  not,  but  that  the  degree  of  impact 
will  depend  on  the  qualitative  as  much  as  the  quantitative  nature  of  that  training. 

5.4.4  Training  approaches  used 

Training  approaches  are  probably  best  formulated  on  the  basis  of  a  task  analysis  (Gael,  1988),  but  there 
seems  little  consensus  in  the  literature  about  how  to  analyse  team  or  collective  tasks  as  a  prelude  to 
design  of  training  programs. 

Any  such  analysis  would  need  to  identify  the  skill  requirements  of  collective  sub-tasks,  their  position 
on  the  critical  path  for  the  collective  task  as  a  whole,  the  interdependence  of  critical  sub-tasks,  and  the 
nature  of  the  individual  and  interactive  skills  required  by  different  positions  in  the  collective.  The 
nature  of  the  equipment  employed  in  the  performance  of  those  tasks,  and  the  range  of  conditions  under 
which  the  collective  task  might  be  conducted  would  also  need  to  be  taken  into  account.  From  this 
analysis  could  be  determined  the  individual,  interactive  and  team  work  knowledge  and  skills  most 
likely  to  affect  collective  performance  and  the  training  approach  best  suited  to  their  acquisition  and 
retention. 

If,  for  instance,  the  collective  task  requires  little  co-ordination,  and  a  lot  of  physical  strength  and 
endurance  on  the  part  of  all  collective  members,  then  retention  of  collective  performance  might  be 
based  primarily  on  rate  of  decay  of  physical  capability  of  the  weakest  member  -  and  the  most 
appropriate  training  approach  to  enhance  physical  capacity  of  all  collective  members. 

Based  on  the  degree  to  which  critical  team  members  (maybe  defined  as  those  performing  critical 
individual,  interdependent,  or  co-ordination  sub  tasks)  had  received  the  most  appropriate  training  for 
skill  acquisition  and  retention,  some  prescription  for  maintenance  or  mission  training  in  terms  of 
objectives  and  approach  might  be  made  for  the  collective  task(s)  that  need  to  be  brought  to  a  state  of 
mission  preparedness. 

Two  levels  of  training  might  be  considered:  maintenance  levels  and  mission  levels.  Maintenance 
levels  might  represent,  for  a  broad  range  of  tasks,  a  spring  board  performance  level  from  which  a 
limited  number  might  need  to  be  brought  to  mission  readiness  within  some  defined  preparation  time. 

Swezey  and  Llaneras  (1977)  cite  research  to  conclude  that  acquisition,  retrieval  and  retention  all 
depend  to  a  large  degree  on  how  the  learning  material  has  been  organized  and  presented.  For  instance, 
conceptually  or  categorically  organized  material  shows  less  memory  loss  and  better  generalisation 
across  different  operational  situations.  Tasks  that  are  easier  to  organize  mentally  are  learned  faster  and 
retained  better  as  does  encouraging  learners  to  relate  new  knowledge  to  past  experiences.  When 
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appropriately  organized  and  customized  to  the  needs  of  the  learner,  even  complex  procedural  tasks  that 
normally  show  lower  retention  may  be  retained  longer.  Instructional  strategies  that  remind  learners  of 
currently  possessed  knowledge  and  relate  this  to  the  new  knowledge,  make  repeated  use  of 
information  presented  from  different  perspectives,  and  encourage  elaboration  of  the  material  during 
the  acquisition  and  the  retention  interval,  are  better  retained. 

Studies  by  Schmidt  and  Bjork  (1992)  suggest  that  while  block  practice  may  enhance  performance  at 
the  end  of  training,  random  practice  shows  better  retention,  even  though  end  of  training  performance 
may  be  reduced.  However,  in  at  least  one  study  of  small  teams,  this  was  not  shown  (Rhodeniser  et  al., 
1998).  In  this  work,  two  person  teams  learned  a  simulated  radar  target  identification  task  using 
massed  or  distributed  practice.  Performance  measures  taken  at  the  end  of  training  and  after  short 
retention  intervals  of  up  to  7  days  showed  that  a  distributed  schedule  resulted  in  better  skill  acquisition 
but  no  difference  in  skill  retention,  suggesting  that  different  practice  schedules  may  work  differently  at 
individual  and  collective  levels.  Nonetheless,  like  much  other  such  work,  this  study  was  conducted 
with  an  artificial  task,  and  with  undergraduate  “teams”  rather  than  true  collectives  used  to  the  domain 
and  to  working  together. 

Techniques  that  permit  learners  to  build  mental  models  of  team  and  task  issues  help  cue  recognition 
and  re-construction  of  information  later  and  are  argued  to  enhance  team  performance  (Cannon-Bowers 
etal.,  1993). 

Training  feedback  or  knowledge  of  results  is  frequently  cited  as  a  key  factor  in  promoting  the 
acquisition  and  retention  of  individual  skills  (Schendel  and  Hagman,  1991).  Feedback  may  facilitate 
skill  retention,  as  it  highlights  successful  performance  and  provides  remedial  information  about 
aspects  of  performance  that  needs  to  be  corrected.  For  collectives,  there  is  good  reason  to  believe  that 
effective  feedback  during  training  will  also  be  a  good  predictor  of  skill  retention.  It  is  important  to 
note,  here,  however,  that  levels  of  feedback  may  be  less  within  collective  training  environments  than 
in  individual  training  environments,  as  the  sheer  number  of  people  is  larger  within  collectives. 
Knowledge  of  results  and  task  feedback  are  seen  as  important  predictors  of  team  performance 
(Tumage  et  al.,  1990). 

One  needs  to  distinguish  between  uses  of  the  terms  cueing  and  feedback(  knowledge  of  results). 

Cueing  or  prompting  may  be  described  an  aid  to  learning  task  sequences  that  should  be  gradually 
withdrawn.  The  term  is  also  used  to  describe  on-the-job  prompts  to  step  a  user  through  the  task.  The 
value  of  prompts  may  vary  with  the  expertise  of  the  performer.  A  first  time  user  may  depend  on  them 
as  may  a  skilled  user  returning  to  the  task  after  a  long  absence.  Skilled  performers  may  find  that 
prompts  interfere  with  multi-tasking  and  slow  down  performance.  Feedback  is  qualitative  information 
about  task  performance  on  a  task  just  performed,  and  may  be  verbal,  sensory,  or  equipment  status 
related.  Longer  post  task  lags  in  feedback  diminish  the  training  benefit  of  feedback,  as  do  too  much  or 
too  little  detail  with  respect  to  the  needs  of  the  learner.  Feedback  can  also  form  a  critical  item  of 
information  for  performance  of  certain  types  of  continuous  or  interactive  tasks,  such  as  tracking 
targets,  or  dovetailing  one  individual  collective  sub-task  with  another. 

Thus,  in  teams,  feedback  becomes  more  complex.  Feedback  (or  cues)  may  be  provided  about  task 
completion  by  others  for  interdependent  tasks,  about  whether  one’s  own  interactive  tasks  are  being 
performed  appropriately  with  respect  to  the  others  in  the  collective  (e.g.  information  being  accurate 
and  timely). 

Although  the  area  of  instructional  strategy  contain  many  inconsistent  or  ambiguous  results,  even  for 
individual  skills,  and  lack  a  comprehensive  theoretical  framework,  nevertheless,  some  training  strategies 
can  be  expected  to  result  in  greater  retention,  at  least  with  respect  to  individual  skills  likely  to  contribute 
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to  collective  performance.  If  the  training  received  by  the  collective  members,  individually  or  together 
could  be  assessed,  then  this  can  be  expected  to  become  an  influential  predictor  of  future  performance  on 
collective  tasks. 

Such  training  approaches  might  include: 

•  Cross  training  collective  members  in  each  others  tasks. 

•  Appropriate  provision  of  feedback  for  key  collective  tasks. 

•  Training  with  other  collective  members  on  collaborative  tasks  that  build  awareness  of 
other  team  members  capabilities. 

•  Conceptual  organization  of  material  to  be  learned. 

•  Using  contextual  interference  to  force  consideration  of  working  in  other  contexts. 

•  Forcing  elaboration  of  cognitive  structures. 

•  Distributed  rather  than  random  practice. 

Predictions  for  retention  of  collective  performance: 

•  The  original  training  approach  used  for  critical  and  interdependent  individual  sub-tasks 
will  affect  retention. 

•  Transfer  from  training  on  other  collective  tasks  during  the  retention  interval  will  affect 
retention. 

5.4.5  Summary  of  Training  Factors 

This  section  considered  the  level  of  learning  after  initial  training;  the  match  between  the  training 
context  and  the  retention  context;  training  received  as  a  collective;  and  training  approaches  used  to 
achieve  collective  performance. 

Discussion  of  these  factors  emphasizes  the  importance  of  understanding  the  contribution  of  individual 
sub-tasks  to  performance  on  the  overall  collective  task.  Achieving  this  understanding  requires  some 
form  of  team  task  analysis  to  assess  sub-task  interdependence,  criticality,  the  need  for  interaction 
among  team  members,  and  the  knowledge  to  be  shared  for  an  effective  team  mental  model.  Then,  the 
knowledge  of  the  effect  of  individual  retention  factors  can  be  focussed  more  diagnostically  to  predict 
retention  of  the  collective  task  as  a  whole.  For  instance  the  degree  of  over-learning,  assessment  of 
learning  strategies  used  to  promote  retention,  transfer  of  training  from  other  collective  tasks  conducted 
during  the  retention  interval,  and  the  degree  to  which  such  collective  training  contributes  to  tacit 
learning  of  knowledge  contributing  to  relevant  components  of  team  mental  models  (such  as  the 
capability  of  other  team  members),  the  implications  of  the  stage  of  learning  reached  (  as  individuals  and 
as  a  collective),  decay  rates  for  different  types  of  procedural  or  knowledge  complexity. 

Assessment  of  both  the  original  the  approach  to  training,  and  related  training  during  the  retention 
interval  could  then  contribute  both  the  prediction  of  collective  retention,  and  suggest  strategies  for 
refresher  training. 
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5.5  Overview  of  Factors 


The  table  below  summarizes  the  factors  discussed  in  this  section  by  category  as  a  preliminary 
framework  for  comparison  of  existing  models  reviewed  in  subsequent  sections  and,  eventually, 
development  of  new  models.  The  columns  on  the  right  of  the  table  refer  to  the  models  to  be  discussed 
in  subsequent  sections. 


Models  reviewed1 

Factors 

N 

ARM 

ARI-2 

R 

Collective 

asks 

ind 

sub-tasks 

1 .  Number  of  collective  tasks  assigned  to  the  unit 

2.  Number  steps  in  individual  sub-tasks 

3.  Availability  of  interstep  cueing  within  sub-tasks 

4.  Predictability  of  sub-task  procedures  &  conditions  of  performance. 

5.  Number  of  discrete  skills  required  by  the  collective  task 

6.  Number  of  sub-units  needed  to  complete  sub-tasks 

7.  Interchangeability  among  collective  sub-tasks 

8.  Interdependence  among  collective  sub-tasks 

9.  Potential  for  error  correction  and  resource  re-deployment. 

10.  Need  for  information  exchange  among  collective  members 

1 1 .  Cognitive  complexity  of  collective  sub-tasks. 

12.  Time  criticality  of  sub-task 

Collective 

structure 

1 .  Structure  of  the  collective  (sub-units  and  levels  of  organization). 

2.  Number  of  people  in  the  collective. 

3.  Leadership  structure 

4.  Potential  for  role  substitution 

Collective 

nember- 

ship 

1 .  Shared  mental  models  of  team  members  and  team  tasks. 

2.  External  turnover  and  internal  turbulence 

3.  Social  environment  in  the  collective. 

ndividual 

Jifferences 

1 .  Aptitudes  of  collective  members 

2.  Diversity  of  experience  among  collective  members 

3.  Experience  in  collective  settings. 

Training 

eceived 

1 .  Level  of  learning  after  initial  training. 

2.  Time  since  training:  retention  interval 

3.  Match  between  the  training  context  and  the  retention  context. 

4.  Training  received  as  a  collective 

5.  Training  strategies  used  to  achieve  collective  performance. 

6.  Acquisition  phase  of  collective  task 

Dther 

1 .  Number  of  items  of  equipment  and  their  complexity 

Table  4:  Summary  of  factors  likely  to  influence  collective  skill  retention. 

=  Nichols;  ARI  =  Army  Research  Institute;  R  =  Roth) 
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Many  diverse  factors  are  likely  to  impact  on  collective  skill  retention.  These  factors  relate  to  the 
characteristics  of  collective  tasks,  features  of  the  collective  itself,  individual  difference  factors,  training 
factors,  and  the  interval  without  training.  It  is  important  to  re-emphasize  that  the  factors  known  to  affect 
retention  of  individual  performance  are  likely  to  affect  retention  of  collective  performance  in  two  ways. 
First,  collective  tasks  will  be  influenced  in  different  ways  by  the  performance  on  individual  sub-tasks. 
Those  sub-tasks  may  be  independent,  interdependent  or  require  skill  at  interaction  with  others.  The 
significance  of  these  sub-tasks  must  be  set  in  the  context  of  the  overall  collective  task,  but  to  a  great 
extent,  retention  can  probably  be  treated  individually.  Second,  and  less  likely,  factors  affecting  retention 
of  individual  tasks  may  work  in  much  the  same  way  on  overall  retention  of  performance  of  collective 
tasks.  Complex  interactive  factors  are  likely  to  complicate  the  picture  even  further  than  in  the  case  of 
retention  of  individual  performance. 

It  remains  to  be  seen,  however,  whether  these  individual  factors  will  show  their  effects  in  the  same  way 
within  a  collective  context.  Positional  factors  will,  for  example,  influence  the  importance  of  individual 
interactive  skills:  e.g.  leadership  or  supervisory  positions  in  the  collective,  or  for  individual  skills 
affecting  tasks  on  the  critical  path  for  the  collective  task. 

The  factors  identified  will  interact  in  complex  ways.  For  example,  the  interaction  between  individual 
aptitude  and  the  level  of  task  intersection  will  have  implications  for  predicted  retention.  For  collectives  in 
which  individual  aptitude  is  highly  variable,  for  example,  a  high  level  of  serial  task  intersection  (that  is, 
sub-tasks  must  be  completed  sequentially)  may  be  particularly  problematic.  As  such,  it  is  important  to 
consider  both  the  features  of  the  task,  and  the  features  of  the  individuals  within  a  collective.  Similarly,  it  is 
also  critical  to  consider  the  entire  organizational  context  of  a  collective  to  understand  and  predict  retention 
of  collective  performance. 

An  emerging  area  of  great  potential  appears  to  be  the  concept  of  team  mental  models.  If  it  is  true,  for 
example,  that  a  superior  team  mental  model  is  related  to  superior  team  performance,  and  that  such 
cognitive  structures  for  knowledge  about  team  procedures  ,  capabilities  of  other  team  members, 
interdependencies  among  sub-tasks,  can  be  readily  trained  and  maintained  without  actually  undertaking 
training  as  an  entire  collective,  this  application  would  appear  to  have  great  potential  not  only  for 
prediction  of  retention  but  as  the  basis  for  some  forms  of  refresher  training  in  the  retention  interval,  and 
even  for  overcoming  the  effects  of  personnel  turnover  and  turbulence. 
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6.  Models  for  Collective  Skill  Retention 


The  purpose  of  this  section  is  to  review  existing  models  and  approaches  relevant  to  the  issue  of 
predicting  collective  skill  retention.  Our  search  yielded  only  one  model  directly  relevant  to  this 
problem,  but  two  other  approaches  that,  although  not  specifically  designed  to  address  the  issue  of 
collective  skill  retention,  either  contribute  important  factors  to  this  purpose  or  may  be  adaptable  to 
predicting  collective  skill  retention.4 

In  this  section,  we  describe  these  models  (or  approaches),  explore  how  each  was  developed,  and 
evaluate  the  extent  to  which  each  of  these  approaches  may  contribute  to  the  prediction  of  collective 
skills.  The  evaluation  considers  the  types  of  skills  to  which  the  model  is  potentially  applicable,  the 
empirical  support  for  the  validity  of  the  model,  the  information  needed  to  use  the  model,  and  the 
practical  issues  associated  with  its  use  (e.g.  time,  cost).  Lastly,  each  model  will  also  be  evaluated 
against  the  categorisation  framework. 

The  section  covers  the  following  topics: 

•  Nichol’s  approach 

•  The  ARI  model 

•  Roth’s  model 

•  Comparison  of  these  models  to  the  categorisation  framework 


6.1  Nichol’s  Approach  to  Predicting  Skill  Retention 

Nichol  (Nichol,  unknown  date  reported  in  Spears,  1991b)  presents  a  simple  and  easy  to  use  approach 
to  predicting  skill  retention  for  both  individual  and  collective  military  tasks.  Although  our  insight  into 
this  model  is  limited  to  the  brief  description  provided  in  a  secondary  reference,  it  is  included  in  the 
review  as  an  important  example  of  an  attempt  to  predict  the  need  for  refresher  training  within  a 
military  context.  Spears  interest  was  in  air  force  ground  defence  tasks.  Although  the  detail  provided 
is  limited,  these  appear  to  be  primarily  individual  tasks  classified  as  not  difficult  or  of  moderate 
difficulty,  such  as  first  aid  or  donning  a  gas  mask.  Some  tasks,  such  as  fire  fighting,  imply  team  work 
but  no  details  are  given. 

6.1.1  Theoretical  Bases 

This  approach  can  be  described  as  atheoretical  as  its  focus  is  mainly  pragmatic.  Its  goal  is  to  estimate 
training  and  refresher  training  priorities  and  frequencies  within  a  military  context,  based  on  task 
difficulty,  importance,  and  normal  frequency  of  performance.  Although  the  development  of  NichoTs 
model  is  unknown,  Spears  (1991b)  indicates  that  the  model  was  based  on  the  ARI  model.  Both  Spears 
and  NichoTs  work  appear  to  address  both  training  and  refresher  training,  but  we  limit  our  discussion 
primarily  to  the  prediction  of  the  need  for  refresher  training. 


4  We  recognize  that  the  NichoTs  approach  is  not  a  model,  in  the  sense  that  it  uses  a  practical  approach,  rather 
than  being  theoretically  grounded.  Nonetheless,  we  have  included  it  within  the  “model”  chapter  as  a  way  of 
representing  the  range  of  approaches  used  to  understand  collective  skill  retention. 
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6.1.2  Using  Nichol’s  Approach 

To  estimate  how  often  tasks  need  refresher  training,  tasks  are  rated  for  difficulty,  importance  and 
frequency  of  performance.  Nichol  defines  each  dimension  and  describes  task  features  for  different 
levels. 

Task  difficulty,  for  example,  is  defined  as  the  amount  of  training  needed  for  the  task  to  be  acquired, 
and  the  amount  of  part-task  training  needed  for  successful  first  performance  of  the  task,  with  tasks 
being  rated  as  easy,  moderately  difficult  or  very  difficult.  To  help  identify  task  factors,  a  descriptor  of 
task  difficulty  is  provided  for  each  level  of  difficulty.  This  descriptor  highlights  factors  seen  in  the 
existing  research  (e.g.  Rose  et  al.,  1985b)  to  contribute  to  individual  skill  retention,  such  as  the 
number  of  steps  in  the  task.  For  example,  very  difficult  tasks  are  described  as  follows: 

Very  Difficult  -  Very  difficult  tasks  are  complex  tasks  with  a  number  of  items  or 
concepts  that  need  to  be  considered  at  any  one  time.  Several  abstract  rules  need  to 
be  followed.  There  are  many  (more  than  10)  independent  steps  that  need  to  be 
performed  in  order.  There  are  several  decision  points  with  numerous  branching. 

There  is  very  close  error  tolerance  requiring  hand-eye  coordination.  There  is  no 
job  aid. 

As  can  be  seen,  this  task  descriptor  addresses  the  number  of  steps  in  the  task  and  a  job  aid  (ranging 
from  a  “memory  jogger”  to  “all  information  necessary”).  Raters  are  asked  to  consider  the  task  in 
question  against  the  task  descriptor,  and  to  rate  the  difficulty  of  the  task  by  choosing  the  descriptor 
which  matches  best  the  task  in  question. 

Task  importance  is  defined  in  terms  of  the  consequence  of  inadequate  or  improper  performance  on  the 
job.  Descriptors  of  importance  are  also  provided  as  examples.  For  example,  the  description  of  a  low 
importance  task  is  as  follows: 

Low  Importance  -  If  the  task  is  not  done,  or  is  done  incorrectly,  it  may  create 
hardship  or  discomfort,  but  will  not  cause  injury >  or  greatly  impair  successful 
performance 

Task  frequency  ratings  are  based  on  the  premise  that  tasks  that  are  performed  more  infrequently  will  be 
more  affected  by  skill  decay.  (Spears,  1991a)  comments  that  frequent  but  incorrect  performance  without 
corrective  feedback  will  have  a  negative  effect  and  affect  not  only  with  future  performance,  but  require 
unlearning  and  corrective  training.)  Task  frequency  ratings  address  whether  a  task  is  performed  very 
often  (at  least  once  every  two  weeks),  moderately  frequently  (at  least  once  every  eight  weeks),  or  not 
frequently  (not  done  for  more  than  8  weeks). 

DIF  ratings  (Difficulty,  Importance  and  Frequency)  are  completed  by  experts  familiar  with  the  task. 

A  matrix  with  27  possible  outcomes  is  then  used  to  convert  DIF  values  into  a  recommendation  for 
training  priorities.  Each  matrix  combination  is  associated  with  a  training  priority  value,  ranging  from 
1  to  6.  Higher  ratings  indicate  a  higher  priority  for  and  a  different  choice  of  six  types  of  refresher 
training. 
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Task  Difficulty 

High  (H) 

Moderate  (M) 

Low  (L) 

Task  Importanc 

High 

1 

Mod 

Low 

High 

1 

Mod 

Low 

High 

I 

Mod 

Low 

Performance  Frequent 

H 

M 

L 

H 

M 

L 

H 

M 

L 

H 

M 

L 

H 

M 

L 

H 

M 

L 

H 

M 

L 

H 

M 

L 

H 

M 

L 

Type  of  training  neede 

2 

1 

1 

2 

2 

1 

3 

3 

2 

3 

2 

2 

3 

2 

2 

4 

3 

5 

4 

4 

3 

4 

4 

3 

5 

6 

6 

Table  5:  Training  Priority  Based  on  Task  Difficulty,  Importance  and  Frequency 

(adapted  from  Nichol,  in  Spears,  1991b) 

Nichol’s  approach  also  provides  an  aid,  used  to  match  the  training  priority  with  a  training  approach. 
This  aid  provides  a  description  of  the  training  needs,  and  lists  the  characteristics  of  tasks  with  which  it 
is  appropriate  to  employ  the  training  approach. 


'raining 

Priority 

Training 

Approach 

Description  of 

Training  Needed 

Tasks  Appropriate 
for  this  Approach 

i 

Compre¬ 

hensive 

A  series  of  3-5  training  sessions  with 
correct  performance  achieved  in  each 
training  session  and  job  proficiency 
attained  by  the  last  session. 

Very  difficult  tasks  of  moderate  to  high  importance, 
tasks  performed  infrequently  or  required  correct 
performance  on  the  job  without  time  to  practice 

2 

Progress 

-ive 

A  series  of  3  training  sessions  with 
correct  performance,  including  job 
proficiency  by  the  last  training 
session. 

Tasks  performed  infrequently,  but  not  as  critical  or 
difficult  as  priority  1  tasks,  or  moderately  difficult  tasks 
that  must  be  performed  correctly  on  the  job  without 
time  to  practice. 

3 

Double 

Session 

Two  separate  training  sessions  with 
correct  performance  achieved  in 
each.  Job  proficiency  is  attained  on 
the  job. 

Tasks  moderately  or  very  difficult  and  important,  and 
performed  on  the  job  at  least  moderately  frequently. 

The  tasks  must  be  learned  in  training,  but  may  be 
mastered  on  the  job. 

4 

Single 

Session 

A  single  training  session  with  the  task 
performed  at  least  on  time  correctly. 

Job  proficiency  is  attained  on  the  job. 

Tasks  moderate  or  less  in  difficulty  and  importance  and 
performed  frequently  on  the  job.  Tasks  are  learned  in 
training  but  may  be  mastered  on  the  job. 

5 

Explain  & 
Demon¬ 
strate 

Tasks  deserve  or  require  explanation, 
but  actual  performance  is  left  for  on 
the  job. 

Very  easy  tasks  of  limited  importance  except  perhaps, 
as  they  relate  to  other  tasks.  They  are  performed  on  a 
moderate  to  frequent  basis  on  the  job  and  may  be 
learned  on  the  job. 

6 

On  the 

Job 

Training  is  accomplished  on  the  job 
on  an  as  needed  basis. 

Very  easy  tasks  of  limited  importance  performed  on  an 
infrequent  basis  on  the  job. 

Table  6:  Description  of  Training  Approaches  (in  Spears,  1991b). 

Lastly,  the  Nichol’ s  model  also  provides  general  guidelines  for  retention  intervals  for  tasks  with 
varying  levels  of  difficulty  in  accordance  with  several  training  strategies,  listed  in  the  table  below. 

Unfortunately,  no  information  is  provided  as  to  how  these  values  were  derived  and  it  is  unclear 
whether  they  are  empirically  based,  or  estimates  of  predicted  skill  retention  over  time.  In  terms  of 
table  interpretation,  the  values  for  a  given  task  and  week  represent  the  predicted  average  performance 
of  the  specific  task.  This  table  can  be  used  to  predict  either  the  probable  performance  on  a  given  task 
after  a  specified  number  of  weeks  have  elapsed,  or  to  predict  the  need  for  refresher  training,  given  a 
specific  proficiency  target. 
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Weeks  since  last  performance 

Training 

Strategy 

Task 

Difficulty 

1 

2 

3 

D 

5 

6 

n 

8 

9 

10 

11 

12 

13 

Single 

session 

very 

94 

89 

84 

79 

75 

71 

67 

63 

60 

56 

53 

50 

47 

moderate 

99 

97 

96 

95 

93 

92 

91 

89 

88 

87 

86 

85 

83 

not 

99 

99 

98 

97 

97 

96 

95 

95 

94 

94 

93 

92 

92 

Double 

session 

very 

96 

93 

89 

86 

83 

79 

76 

74 

71 

68 

66 

63 

61 

moderate 

99 

99 

98 

97 

97 

96 

95 

95 

94 

93 

93 

92 

91 

not 

100 

99 

99 

99 

99 

98 

98 

98 

98 

97 

97 

97 

96 

Pro 

-gressive 

very 

97 

94 

92 

89 

87 

84 

82 

79 

77 

75 

73 

71 

69 

moderate 

99 

99 

98 

97 

97 

96 

95 

95 

94 

93 

93 

92 

91 

not 

100 

99 

99 

99 

98 

98 

98 

97 

97 

97 

96 

96 

96 

Compre 

-hensive 

very 

98 

95 

93 

91 

89 

87 

85 

83 

81 

79 

77 

75 

73 

moderate 

99 

99 

98 

98 

97 

97 

96 

95 

95 

94 

94 

93 

93 

not 

100 

99 

99 

99 

99 

98 

98 

98 

98 

97 

97 

97 

96 

Weeks  since  last  performance 

Training 

Strategy 

Task 

Difficulty 

14 

15 

16 

17 

18 

19 

20 

21 

22 

23 

24 

25 

26 

Single 

session 

very 

45 

42 

40 

38 

36 

34 

32 

30 

28 

27 

25 

24 

22 

moderate 

82 

81 

80 

79 

78 

77 

76 

75 

74 

73 

72 

71 

70 

not 

91 

90 

90 

89 

89 

88 

88 

87 

86 

86 

85 

85 

84 

Double 

session 

very 

58 

56 

54 

52 

50 

48 

46 

45 

43 

41 

40 

38 

37 

moderate 

88 

87 

86 

85 

85 

84 

83 

82 

82 

81 

80 

79 

79 

not 

94 

94 

93 

93 

92 

92 

91 

91 

91 

90 

90 

89 

89 

Pro 

-gressive 

very 

67 

65 

63 

61 

60 

58 

56 

55 

53 

52 

50 

49 

47 

moderate 

91 

90 

89 

89 

88 

88 

87 

86 

86 

85 

85 

84 

83 

not 

95 

95 

95 

94 

94 

94 

94 

93 

93 

93 

92 

92 

92 

Compre 

-hensive 

very 

72 

70 

68 

67 

65 

63 

62 

60 

59 

58 

56 

55 

54 

moderate 

92 

92 

91 

91 

90 

90 

89 

89 

88 

88 

87 

87 

86 

not 

96 

96 

96 

95 

95 

95 

95 

94 

94 

94 

94 

93 

93 

Table  7:  Predicted  Skill  Retention  Over  Time  (in  Spears,  1991b). 

6.1.3  Applicability  and  Practicality 

There  is  little  evidence  from  the  available  literature  related  to  how  this  model  has  been  used  to  predict 
skill  retention.  Spears  (1991b)  does  use  this  model  to  estimate  the  proficiency  on  ground  defense  tasks 
(Table  8)  for  up  to  one  year  after  training.  An  important  implication  of  the  Spears  review  is  that 
prediction  of  retention  needs  to  be  considered  together  with  the  type  of  training  to  be  provided.  Spears 
describes  six  training  approaches  ranging  from  on  the  job  feedback  to  systematic  multi-session  training. 
Spears  advocates  individual  testing  prior  to  commencement  of  any  form  of  training  to  confirm  that 
performance  is  indeed  incorrect  before  continuing  with  any  training.  The  assumption  is  made  that  the 
tasks  of  interest  will  continue  to  be  performed  over  the  period  in  question  to  some  degree,  resulting  in 
consideration  of  performance  frequency  as  a  factor. 
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GDT  Tasks 

Task  components 

Task  Difficulty 

Weapon  skills 

Weapon  handling 

Moderately  difficult 

Rules  of  engagement 

Very  difficult 

Field  craft 

Not  difficult 

First  Aid 

Not  difficult 

NBC 

Warnings 

Not  difficult 

Personal  Drills 

Moderately  difficult 

Shelter  Drills 

Not  difficult 

Post  attack  recovery 

Reporting 

Moderately  difficult 

Searching 

Moderately  difficult 

Firefighting 

Moderately  difficult 

lEDs/UXB 

Moderately  difficult 

Table  8:  Ground  Defence  Tasks  (GDT)  (Spears,  1991b) 

These  tasks  were  then  used  to  rate  the  common  core  GDT  tasks  and  their  components.  As  the 
Nichol’s  retention  table  only  extends  to  14  weeks,  b  (1991)  extended  the  prediction  tables  to  weeks  28 
to  52,  assuming  a  linear  decline  in  performance.  Note  that  Spears  (1991b)  appears  to  include  both 
individual  and  collective  tasks  with  no  distinction  between  them.  This  suggests  that  it  is  theoretically 
possible  to  use  the  NichoTs  approach  with  all  kinds  of  tasks.  Unfortunately,  Spears  (1991b)  provides 
no  additional  detail  about  how  the  estimates  derived  from  NichoTs  approach  were  used,  but 
presumably  they  were  used  to  predict  task  proficiency  after  intervals  without  training.  Spears 
recommends  medium  (four  month)  and  longer  term  validation  studies  to  evaluate  the  NichoTs  model, 
but  references  to  such  work  were  not  found.  As  a  whole,  then,  there  appears  to  be  very  little  work 
available  on  the  validity  of  the  NichoTs  approach  to  predict  the  retention  of  skill,  collective  or 
otherwise. 

Nonetheless,  this  approach  represents  a  simple  and  very  low  cost,  paper  and  pencil  measure  that 
requires  little  training  to  administer.  NichoTs  approach  is  sufficiently  broad  that  it  can  be  used  with 
virtually  any  task  that  can  be  specified.  As  the  success  of  this  model  is  predicated  on  the  ability  to 
segment  tasks,  it  remains  to  be  seen  how  easily  more  complex  collective  tasks  (e.g.  command  and 
control  tasks)  could  be  dealt  with. 

Despite  its  ease  of  use  and  its  wide  range  of  applicability,  however,  it  seems  unlikely  that  such  an 
approach,  on  its  own,  has  the  sophistication  needed  to  predict  collective  skill  retention.  Identifying 
task  difficulty  as  an  important  predictor  of  skill  retention,  of  course,  is  not  a  unique  contribution  of  this 
model,  but  other  conceptualisations  of  task  difficulty  or  complexity  (the  UDA  model,  for  example) 
allow  less  room  for  misinteipretation  by  raters.  Moreover,  although  it  seems  clear  that  task  difficulty, 
importance  and  frequency  may  well  play  a  role  in  collective  skill  retention,  it  seems  unlikely  that  they 
are  the  most  important  predictors  of  collective  skill  loss,  or  that  their  description  as  sub-tasks  in 
relation  to  the  overall  collective  tasks  cannot  be  improved.  Most  importantly,  the  NichoTs  approach 
overlooks  the  nature  of  the  collective,  the  training  environment,  and  many  other  characteristics  of 
collective  tasks  that  are  likely  to  impact  on  skill  retention.  It  is  possible  that  modification  of  NichoTs 
evaluation  scales  to  include  collective  aspects,  such  as  task  interdependence,  might  permit  extension  to 
collectives  and  their  tasks  -  but  this  is  doubtful. 

Perhaps  NichoTs  most  important  contribution  is  to  specify  the  importance  of  the  task,  in  terms  of  the 
real  life  consequences  if  the  task  is  not  completed  properly.  This  factor  seems  a  critical  one  within 
military  contexts  with  limited  resources,  and  consideration  of  the  importance  of  the  task  should  be 
incorporated  into  ratings  of  collective  refresher  training  priority.  Considering  the  frequency  with 


Human systems®  Incorporated 


Collective  and  Cognitive  Skill  Retention 


Page  57 


which  the  task  is  normally  performed  is  also  somewhat  unique  and  underscores  the  difficulty  of 
defining  the  retention  interval  as  being  a  period  during  which  no  practice  whatsoever  occurs. 

In  conclusion,  although  this  model  does  not  seem  complex  enough  to  warrant  further  advancement, 
task  importance  and  frequency  are  concepts  to  consider  for  future  models  of  collective  skill  retention, 
but  with  better  measures  of  each  of  these  dimensions. 

6.2  The  ARI  Model  of  Skill  Retention5 

The  most  fully  developed  model  designed  to  predict  skill  retention  at  the  individual  level  is  the  User’s 
Decision  Aid  (UDA),  developed  by  the  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social 
Sciences  (ARI)  (Rose  et  al.,  1985a;  Rose  et  al.,  1985b).  This  model  was  developed  specifically  to 
provide  quantitative  predictions  of  skill  retention  for  individual  military  tasks. 

Although  the  UDA  model,  as  originally  conceived,  focused  mainly  on  procedural  skills,  rather  than  on 
the  broader  range  of  continuous  and  even  cognitive  skills,  subsequent  research  has  worked  to  extend 
the  range  of  individual  skills  predicted.  Because  of  the  prominence  of  this  model  within  the  skill 
retention  literature,  its  ease  of  use,  and  its  promise,  this  research  is  reviewed  in  the  following  sections, 
with  a  view  toward  explored  the  possibilities  of  extending  the  model  further,  to  collective  skill 
retention. 

6.2.1  Theoretical  Bases 

The  UDA  model  is  based  on  years  of  research  conducted  by  the  ARI.  As  researchers  uncovered  the 
personal,  task,  training,  and  job  condition  factors  that  affect  skill  retention,  they  began  to  work  on 
descriptive  procedures  to  identify  more  precisely  how  these  factors  produced  deterioration  of  skills 
(e.g.,  Hurlock  &  Montague,  1982).  The  ARI  established  their  modelling  effort  to  develop  and  validate 
a  convenient,  practical  method  for  unit  commanders  and  training  managers  to  decide  upon  the 
allocation  of  training  resources  to  maximize  combat  readiness  (Rose  et  al.,  1985b). 

The  first  issue  dealt  with  in  the  ARI  initiative  was  deciding  which  factors  were  relevant  to  prediction  of 
individual  skill  loss  as  well  as  feasible  for  use  in  prediction.  Since  testing  large  number  of  soldiers  on  each 
Aimy  task  to  derive  empirical  individual  retention  curves  would  be  extremely  resource  intensive,  the  ARI 
instead  used  the  factors  already  identified  in  the  research  literature  to  develop  theoretical  retention  curves 
for  use  in  prediction  of  skill  loss  (Rose  et  al.,  1985b). 

Several  ARI  projects  had  investigated  a  broad  range  of  factors  that  impact  on  the  ability  of  individuals 
to  retain  previously  acquired  skills  over  extended  periods  of  time.  This  program  of  research  explored 
the  effects  of  various  training  variables,  such  as  overlearning  (Hagman  &  Rose,  1983),  and  practice 
schedules  (Schendel  &  Hagman,  1982).  In  addition,  the  program  explored  the  effects  of  individual 
abilities  on  the  retention  of  skills,  and  concluded  that  the  effects  of  individual  ability  measures  were 
inconsistent  (Rose  et  al.,  1985b).  These  ARI  studies  also  identified  a  number  of  task  characteristics 
affecting  skill  retention  such  as  task  difficulty,  the  number  of  steps  required  to  complete  the  task, 
interstep  cueing,  and  step  relevance  (Rose  et  al.,  1985b). 

As  the  result  of  this  work,  ARI  researchers  decided  that  factors  related  to  training  and  individual 
differences  presented  unacceptable,  practical  problems  of  assessment,  and  chose  to  focus  on  task  factors 
as  predictors  of  skill  retention.  Work  by  Wisher  et  al.  (1999)  reviews  the  major  task  factors  identified 
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and  used  in  the  ARI  model,  summarized  in  the  table  below:  These  task  factors  were  the  starting  point 
for  development  of  the  UDA  model. 


Task  Factors  Identified  by  the  ARI 

1. 

Number  of  steps 

2. 

Whether  steps  must  be  performed  in  a  set  sequence 

3. 

Whether  the  task  contains  feedback  that  indicates  the  correct  order  of  steps 

4. 

Number  of  facts  or  information  chunks  that  must  be  recalled 

5. 

Execution  demands 

6. 

Whether  the  skill  is  cognitive  or  perceptual/motor 

7. 

Whether  there  are  job  and/or  memory  aids  for  the  task 

8. 

The  time  limit  of  the  task  (if  any) 

Table  9:  Task  Factors  Identified  During  Development  of  the  UDA  Model 
of  Skill  Retention 

6.2.2  Development  of  the  Model 

The  ARI  model  of  skill  retention  was  designed  to  address  the  issue  of  how  rapidly  individual  tasks, 
once  learned,  are  forgotten  over  intervals  without  practice.  The  major  goal  of  the  ARI  project  was  to 
determine  a  way  to  keep  soldiers  proficient  in  tasks  (Rose  et  al.,  1985a).  Given  that  forgetting  will 
occur,  the  ARI  model  concentrated  on  developing  a  tool  to  increase  the  efficiency  of  scheduling 
refresher  training.  This  tool  would  enable  accurate  prediction  of  which  tasks  are  likely  to  drop  below 
a  criterion  level  of  performance,  so  that  only  necessary  refresher  training  could  be  scheduled.  The 
measure  used  by  the  UDA  model  was  the  percentage  of  soldiers  in  the  study  who  were  at  or  above  the 
criterion  performance  level  and  the  percentage  of  task  steps  for  which  soldiers  were  at  or  above 
criterion  performance. 

The  UDA  was  developed  according  to  a  few  basic  steps  (Rose  et  al.,  1985b): 

1 .  Identify  task  dimensions  most  likely  to  be  related  to  retention. 

2.  Convert  task  dimensions  into  rating  scales,  develop  anchor  points,  and  analytically 
assign  weights  to  each  point  on  the  scales. 

3.  Assess  each  scale’s  reliability  and  validity  by  having  several  judges  rate  tasks 
on  each  scale. 

4.  Examine  inter-rater  agreement  and  correlation  between  task  ratings  and  actual 
retention  data. 

5.  Iterate  these  steps  to  develop  a  set  of  valid  and  reliable  scales. 

Rose  et  al.  (1985b)  describe  each  of  these  steps  in  detail,  effectively  laying  out  a  “cookbook”  for 
development  of  a  model  of  skill  retention.  The  remainder  of  this  section  will  briefly  describe  the 
methodology. 

The  basis  of  the  UDA  model  is  the  algorithm  that  weights  and  summarizes  the  relevant  task  factors 
(Table  9  above)  to  produce  a  single  task  retention  score.  This  score  is  then  used  to  predict  soldiers’  level 
of  task  proficiency  over  time.  ARI  researchers  developed  the  initial  algorithm  from  the  empirical 
literature  available.  To  refine  the  algorithm,  researchers  then  performed  regression  analyses  on  data 
obtained  from  previous  ARI  studies.  The  regression  equations  obtained  for  the  set  of  eight  task  variables 
indicated  substantial  correlations  between  retention  at  two  months  among  the  factors.  Correlations 
between  retention  at  four  months  and  the  factors  were  not  as  high  but  still  significant.  Thus,  the  initial 
algorithm  was  deemed  to  provide  a  viable  means  to  predict  skill  retention. 
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To  refine  the  algorithm  further,  ARI  researchers  collected  performance  data  from  soldiers  for  a  sample 
of  13B10  (cannon  crewman)  tasks,  according  to  the  Military  Occupational  Speciality  (MOS) 
definitions.  These  data  were  used  to  establish  empirical  acquisition  and  retention  functions 
(performance  plotted  as  a  function  of  time)  and  to  examine  the  relationships  of  task  factors  to  retention. 
The  MOS  tasks  used  in  this  phase  of  the  project  were  all  basic  skills.  A  total  of  1 1  tasks  could  be  tested 
within  the  constraints  of  the  study. 

Based  on  these  early  studies,  researchers  were  able  to  develop  and  implement  the  UDA  procedure  and 
to  determine  values  for  each  rating  option  of  the  UDA  questions.  This  research  also  improved  the 
comprehensibility  of  the  rating  procedure  based  on  user  feedback.  The  ARI  researchers  had  to  create 
the  preliminary  model  using  arbitrary  assumptions  rather  than  deriving  scale  values  empirically. 
Furthermore,  the  relative  weights  for  rating  scales  and  the  rule  for  combining  them  were  also  arbitrary. 
The  researchers  did  this  because  they  had  not  had  access  to  a  sufficient  pool  of  tasks  to  conduct 
regression  analyses  necessary  to  determine  best- fit  values. 

To  increase  the  number  of  cases,  researchers  combined  data  sets  from  multiple  former  studies  to  create 
a  data  set  containing  data  on  a  total  of  54  tasks.  To  decrease  the  number  of  task  characteristic 
predictors,  they  performed  a  factor  analysis  of  the  task  dimensions  comprising  the  UDA,  which 
reduced  the  number  of  independent  predictors  to  five.  Eight  raters  then  made  ratings  for  the  54  tasks. 
From  the  data  set,  the  researchers  determined  the  percentage  of  soldiers  in  the  study  who  were  at  or 
above  the  criterion  performance  level  (i.e.  GO)  and  the  percentage  of  task  steps  for  which  soldiers 
were  at  or  above  criterion  performance  (i.e.  GO6).  Analyses  resulted  in  empirically-based  scale  values 
for  the  answer  options  within  each  UDA  rating  question.  This  scaling  procedure  is  critical  to  convert 
the  retention  level  predicted  in  the  model  into  a  value  for  “real-world”  retention.  It  also  provided 
regression  weights  for  each  UDA  question  to  account  for  differences  in  the  importance  of  different 
factors. 

We  can  break  down  this  procedure  into  five  more  specific  steps. 

The  first  step  involved  creating  a  matrix  of  the  raters’  modal  responses  arrayed  by  eight  UDA  factor 
questions  (Table  9  above)  and  the  54  tasks.  Then,  researchers  input  the  mode  matrix  into  an  analysis 
of  variance  (ANOVA)  and  conducted  separate  ANOVAs  for  each  of  the  eight  UDA  questions,  with 
percent  soldiers  GO  as  the  dependent  variable  and  answer  options  for  each  question  as  the  independent 
variables.  They  then  generated  cell  means  for  each  answer  option  of  five  of  the  eight  questions.  The 
remaining  three  questions  had  non-significant  F  values  and  so  weights  were  assigned  arbitrarily. 

In  the  second  step,  researchers  then  transformed  the  matrix  of  modal  responses  by  replacing  each  entry 
with  the  corresponding  cell  mean  from  the  ANOVA.  Thus,  the  matrix  values  corresponded  to  the  mean 
percentage  of  soldiers  GO  for  all  tasks  with  a  given  question  option.  For  example,  this  would  change  a 
rating  of  “2”  in  the  question  pertaining  to  the  presence  of  job  aids  to  “61.3,”  which  was  the  mean  percent 
of  soldiers  GO  for  all  tasks  with  “Good”  job  aids.  The  factor  analysis  indicated  that  four  questions  which 
could  be  interpreted  as  the  cognitive  demands  of  the  task  loaded  significantly  onto  one  factor.  Thus, 
researchers  combined  these  questions  into  a  single  independent  variable  by  adding  values,  reducing  the 
original  eight  factors  or  questions  to  five  variables  of  interest. 

The  third  step  was  to  multiply  matrix  values  by  the  question  weights.  The  regression  analysis 
examined  the  relationship  between  five  UDA  variables  and  mean  two-month  retention  performance 
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across  the  54  tasks.  The  results  indicated  that  the  five  predictor  variables  accounted  for  79%  of  the 
variance  in  performance. 

The  fourth  step  involved  transforming  the  scale  values.  The  weights  were  rounded  off  to  the  nearest 
whole  number  and  a  constant  added  or  subtracted  for  each  UDA  question  so  that  the  lowest  value 
equalled  zero.  The  sum  of  the  additions  and  subtractions  were  added  to  the  regression  equation 
constant  so  that  the  transformations  had  no  effect  on  the  regressions  but  improved  the  usability  of  the 
scales. 

The  final  step  was  to  add  up  the  eight  transformed  scores  for  each  task  to  arrive  at  a  total  UDA  score 
for  each  task. 

The  purpose  of  the  UDA  model  is  to  generate  numerical  predictions  of  skill  retention.  Thus,  an 
important  issue  is  how  to  relate  predictions  of  the  model  to  actual  performance;  in  other  words,  what 
does  a  UDA  value  of,  say,  86  mean  and  how  can  it  be  used  to  estimate  performance  (Rose  et  al., 

1985b)? 

The  translation  of  UDA  value  for  predicting  two-month  retention  performance  is  straightforward. 
Because  the  option  weights  and  combination  rule  were  developed  from  data  on  two-month  retention 
performance;  a  UDA  value  of  86  is  a  prediction  of  86%  of  40  (the  constant  in  the  regression  equation) 
or  26%  proficiency  after  two  months.  In  other  words,  26%  of  soldiers  in  a  unit  will  be  able  perform 
the  task  to  the  required  criterion  two  months  after  receiving  training  in  a  given  skill. 

ARI  researchers  also  developed  a  best  fit  generative  function  to  allow  prediction  at  any  retention 
interval.  The  best  fitting  function  was: 

p  =  100*  (UDA/ 1 00)Y/8 

where  p  is  the  predicted  proficiency  and  y  is  the  retention  interval  in  weeks. 

Thus,  retention  performance  after  y  weeks  of  no  practice  is  predicted  by  dividing  the  UDA  value  (after 
subtracting  the  constant  i.e.  40)  by  100,  taking  the  eighth  root,  raising  the  result  to  the  yth  power  (i.e. 
the  number  of  weeks  without  practice),  then  multiplying  by  100.  At  two  months,  y  =  8,  and  the 
formula  reduces  to  just  the  UDA  score.  Researchers  then  turned  to  improving  the  usability  of  the 
UDA.  Actual  and  potential  users  of  the  model  were  interviewed.  Based  on  this  feedback,  researchers 
revised  the  questions  and  instructions  to  improve  comprehensibility  and  convenience  of  the  UDA  as  a 
tool  for  predicting  skill  retention. 

6.2.3  The  UDA  Decision  Aid 

The  UDA  model  was  developed  to  form  a  decision  aid  tool  to  be  used  in  predicting  skill  retention  and 
the  need  for  refresher  training  in  military  tasks  (Rose  et  al.,  1985a).  Specifically,  the  UDA  is  used  to 
determine  (Rose  et  al.,  1985a;  Wisher  et  al.,  1999): 

•  How  quickly  task  skills  will  be  forgotten. 

•  Which  task  among  several  will  be  forgotten  or  remembered  after  a  specified  interval. 

•  What  percent  of  soldiers  will  be  able  to  perform  a  task  after  up  to  one  year  without 
practice. 

•  When  to  conduct  refresher  training  to  keep  a  group  at  a  criterion  level. 

The  UDA  employs  ratings  of  task  characteristics  to  predict  skill  retention.  The  ratings  are  made  by 
individuals  with  extensive  knowledge  of  the  task.  Optimally,  more  than  one  person  performs  task  ratings 
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to  ensure  accuracy  (Rose  et  al,  1985a).  Differences  in  ratings  must  be  resolved  by  discussion  because  the 
UDA  accepts  only  a  single  value  for  each  rating.  All  that  is  required  to  make  task  ratings  is  a  task 
summary  that  provides  descriptive  information  that  can  help  resolves  differences  of  judgment  and  reduce 
the  subjectivity  of  ratings  (Rose  et  al.,  1985a).  If  no  task  summary  is  available,  having  raters  write  their 
own  summary  can  help  them  be  more  thorough  and  objective. 


Question 

Scale  Values 

Factor 

1 .  Are  job  or  memory  aids  used  by 
the  soldier  in  performing  (and  in 
the  performance  evaluation  of)  this 
tasks? 

1  (yes),  0  (no) 

Task  Characteristic; 
Presence  of  job  aid 

2.  How  would  you  rate  the  quality  of 
the  job  or  memory  aid? 

56  (excellent),  25  (very  good),  2  (marginally  good),  1  (poor) 

Task  Characteristic; 
Presence  of  job  aid 

3.  Into  how  many  steps  has  task 
been  divided? 

25  (one),  14  (two  to  five),  12  (six  to  ten),  0  (more  than  ten) 

Task  Characteristic; 
Number  of  steps 

4.  Are  the  steps  in  the  task  required 
to  be  performed  in  a  definite 
sequence? 

10  (None),  5  (all),  0  (some  are  and  some  are  not) 

Task  Characteristic; 
Organization 

5.  Does  task  provide  built-in 

feedback  so  that  you  can  tell  if  you 
are  doing  each  step  correctly? 

22  (for  all  steps),  19  (for  greater  than  50%  of  steps), 

1 1  (for  up  to  50%  of  steps),  0  (for  none) 

Task  Characteristic; 
availability  of 
feedback 

6.  Does  the  task  or  part  of  the  task 
have  a  time  limit  for  its 
completion? 

40  (no  time  limit),  35  (easy  time  limit),  0  (difficult  time  limit) 

Task  Characteristic; 
Stress 

7.  How  difficult  are  the  mental 
processing  requirements  of  this 
task? 

37  (almost  no  requirements),  28  (simple  requirements), 

3  (complex  requirements),  0  (very  complex  requirements) 

Task  Characteristic; 
Difficulty 

8.  How  many  facts,  terms,  names, 
rules,  or  ideas  must  a  soldier 
memorize  to  do  task? 

20  (none),  18  (one  to  three),  13  (four  to  eight),  0  (more  than  eight) 

Task  Characteristic; 
Difficulty 

9.  How  hard  are  the  facts,  terms,  that 
must  be  remembered? 

34  (not  applicable;  none  to  remember),  31  (not  hard  at  all), 

12  (somewhat  hard),  0  (very  hard) 

Task  Characteristic; 
Difficulty 

10.  What  are  motor  control  demands 
of  task? 

2  (none),  0  (small),  16  (considerable),  3  (very  large) 

Task  Characteristic; 
Difficulty 

Table  10:  UDA  Questions  and  Scale  Values  formatting 

The  UDA  contains  ten  questions  that  raters  answer  based  on  the  task  summary  and  their  knowledge  of 
the  task.  Each  question  has  a  set  of  answers  to  choose  from.  Raters  select  the  appropriate  answer  and 
note  the  scale  value  associated  with  the  selected  answer.  The  questions  and  their  scale  values  are 
listed  in  Table  10.  When  all  ten  questions  have  been  answered,  the  raters  compute  the  total  of  the 
scale  values,  which  constitutes  the  task’s  retention  score.  Multiple  raters  review  the  ratings  given  to 
the  task  and  resolve  differences  to  produce  a  final,  agreed-upon  task  retention  score. 

The  UDA  contains  two  Performance  Prediction  Tables  that  are  used  to  convert  the  total  retention  score 
into  a  prediction  of  performance  for  the  rated  task.  The  numbers  in  the  tables  represent  the  expected 
proportion  of  soldiers  in  a  unit  that  are  able  to  perform  the  task  correctly  at  various  retention  intervals 
up  to  one  year  without  practice.  The  UDA  does  not  predict  retention  performance  for  individuals. 

The  first  table  plots  the  expected  proportion  of  soldiers  versus  monthly  intervals,  whereas  the  second 
table  plots  the  expected  proportion  of  soldiers  versus  weekly  intervals. 
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Predicted  %  of  Soldiers  at  Criterion  Performance  Level 

Months  Since  Last  Performance 

Total  Score 

1 

2 

3 

4 

5 

6 

7 

8 

180+ 

100 

100 

100 

100 

100 

100 

100 

100 

175 

97 

95 

92 

90 

87 

85 

83 

81 

170 

94 

90 

85 

81 

76 

72 

69 

65 

165 

92 

85 

78 

72 

66 

61 

56 

52 

160 

89 

80 

71 

64 

57 

51 

45 

40 

155 

86 

75 

64 

56 

48 

42 

36 

31 

150 

83 

70 

58 

49 

40 

34 

28 

24 

145 

80 

65 

52 

42 

34 

27 

22 

17 

140 

77 

60 

46 

36 

27 

21 

16 

12 

135 

74 

55 

40 

30 

22 

16 

12 

9 

130 

70 

50 

35 

25 

17 

12 

8 

6 

125 

67 

45 

30 

20 

13 

9 

6 

4 

120 

63 

40 

25 

16 

10 

6 

4 

115 

59 

35 

20 

12 

7 

4 

110 

54 

29 

16 

8 

4 

105 

50 

25 

12 

6 

100 

44 

20 

8 

95 

38 

15 

90 

31 

Table  11:  UDA  Performance  Prediction  Table  (Month  Intervals) 

To  find  the  percentage  of  soldiers  expected  to  be  able  to  perform  the  rated  task,  raters  find  the  table 
entry  corresponding  to  the  total  retention  score  and  the  retention  interval  of  interest. 

•  To  determine  how  frequently  refresher  training  is  needed,  raters  find  the  row 
corresponding  to  the  total  retention  score  then  locate  the  table  entry  of  50%  (or  any  other 
criterion).  The  retention  interval  at  which  the  criterion  occurs  is  the  maximum  interval  for 
refresher  training  to  sustain  a  unit  at  the  criterion  level. 

•  The  level  of  proficiency  expected  for  a  unit  that  receives  refresher  training  at  a  given 
interval  of  time  (e.g.,  every  X  months)  is  given  by  the  table  entry  corresponding  to  the 
total  retention  score  and  the  interval  (X  months). 

6.2.4  Empirical  Evaluation 

Many  studies  cited  in  support  of  the  UDA  (such  as  Macpherson  et  al.,  1989;  Sabol  et  al.,  1990;  Wisher 
et  al.,  1991)  did  not  actually  compare  predicted  retention  performance  to  actual  retention  performance. 
As  such,  they  do  not  provide  a  true  validation  of  the  UDA  because  the  accuracy  with  which  the  UDA 
predicts  skill  retention  cannot  be  determined. 

Only  one  study  was  found  to  report  a  comparison  between  UDA  predictions  and  actual  retention  data. 
Rose  et  al.  (1985b)  used  data  collected  from  a  sample  of  22  13B10  (cannon  crewman)  MOS  tasks  that 
had  not  been  used  in  the  development  of  the  UDA  to  compare  predictions  of  the  UDA  to  actual  skill 
loss.  Each  of  these  22  tasks  was  rated  by  5  task  experts  on  the  10  UDA  task  dimensions.  Inter-rater 
correlation  on  the  task  ratings  was  quite  high,  with  an  average  correlation  of  more  than  0.9,  suggesting 
good  agreement  on  the  task  ratings  for  the  22  tasks.  Soldiers  were  trained  to  100%  proficiency  on  the 
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tasks,  and  performance  measures  were  taken  at  intervals  of  2,  5  and  7  months.  In  terms  of  the 
percentage  of  soldiers  “GO”,  the  correlation  between  the  actual  performance  and  the  predicted 
performance  was  significant  at  all  3  intervals,  with  the  best  correlation  (r  =  .90)  at  the  2  month 
interval.  At  the  5  and  7  month  intervals,  there  was  an  increasing  tendency  toward  underestimating  the 
prediction  scores.  Similarly,  in  terms  of  the  number  of  steps  “GO”  for  each  task,  the  correlation 
between  actual  and  predicted  performance  at  all  3  time  intervals  was  also  significant,  ranging  from  .83 
at  2  months  to  .70  at  7  months.  This  work  is  the  only  available  empirical  evaluation  of  the  UDA 
model,  but  it  does  provide  evidence  that  the  UDA  model  can  predict  the  retention  of  individual  skills. 

No  research  was  found  that  used  the  UDA  model  to  predict  retention  of  collective  tasks. 

6.2.5  Applicability  and  Practicality 

There  is  good  evidence  for  the  applicability  and  practicality  of  the  UDA  procedure  for  predicting 
individual  task  retention,  as  it  has  been  used  fairly  extensively  for  predicting  the  retention  of  individual 
skills.  The  UDA  model  has  been  used  to  predict  skill  retention  on  several  vehicle  maintenance  tasks 
(Macpherson  et  al.,  1989)  and  has  shown  itself  to  be  relatively  easy  to  use.  Instructors’  task  ratings  were 
completed  in  only  about  12  minutes  per  task,  with  instructors  showing  little  difficulty  in  reaching 
agreement  on  task  ratings.  Moreover,  theoretical  retention  curves  were  also  completed  for  several  of 
these  tasks,  which  allowed  the  prediction  of  the  task  characteristics  most  likely  to  impact  on  skill  decay 
within  this  set  of  vehicle  maintenance  tasks.  This  work,  although  theoretical,  speaks  to  the  ease  with 
which  the  UDA  approach  can  be  used  to  understand  skill  retention  within  a  given  task  domain. 

Predictions  of  retention  using  “peace  support  operation”  tasks  (Wisher  et  al.,  1996)  have  also  been 
formed  using  the  UDA  model.  As  in  the  Sabol  et  al.  (1990)  study,  SMEs  rated  a  number  of  tasks  and 
these  ratings  served  as  a  basis  for  computing  skill  retention  curves  using  the  UDA.  Although  no 
comparison  to  actual  retention  performance  was  made,  the  UDA  predictions  were  sufficiently  precise 
to  allow  researchers  to  rank  order  tasks  in  terms  of  their  expected  difficulty  and  susceptibility  to 
forgetting.  This  rank  order  was  distributed  in  a  guide  job  aid  to  Army  trainers  for  Operation  Joint 
Endeavor  in  Bosnia. 

These  studies  demonstrate  that  it  is  feasible  to  use  the  UDA  to  differentiate  and  prioritize  individual 
tasks  within  a  specific  domain.  Raters  were  able  to  identify  those  tasks  most  susceptible  to  skill  decay 
and  make  predictions  of  when  refresher  training  would  be  needed.  Both  studies  also  demonstrate  the 
practicality  of  the  UDA.  The  use  of  the  UDA  is  a  cost-effective  supplement  to  field  studies  that 
requires  few  materials  and  relatively  little  effort.  In  addition,  the  resolution  procedure  of  the  UDA 
seems  to  work  well  and  allows  multiple  raters  to  achieve  a  single  retention  score.  The  UDA  has  been 
demonstrated  to  be  relatively  easy  to  administer.  Some  training  is  needed  to  perform  the  ratings 
correctly  and  reliably  but  this  training  requirement  does  not  seem  to  be  excessively  burdensome 
(Radtke  &  Shettel,  1985).  As  a  paper  and  pencil  measure,  the  UDA  itself  is  low-cost  and  requires  no 
special  equipment. 

Despite  its  apparent  applicability,  however,  the  UDA  model  as  originally  conceived  has  a  number  of 
potential  limitations  in  addressing  collective  and  cognitive  skill  retention.  The  most  obvious  limitation 
is  that  it  was  originally  designed  for  tasks  which  are  highly  procedural,  as  opposed  to  primarily 
cognitive  or  continuous  motor  tasks  (Sabol  and  Wisher,  2001).  As  most  collective  tasks  are  comprised 
of  both  cognitive  and  continuous  motor  tasks,  the  original  UDA’s  ability  in  this  respect  may  be 
somewhat  limited. 

More  recent  work,  however,  has  extended  the  UDA  model’s  range  of  applicability  to  a  broader  set  of 
skills.  Although  this  work  addresses  the  measurement  of  only  some  cognitive  skills,  it  represents  a 
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significant  step  forward  in  terms  of  thinking  about  how  to  model  collective  skill  retention.  Two 
studies  have  attempted  to  address  more  cognitive  skills,  and  have  used  the  UDA  model  in  an  unaltered 
and  considerably  altered  form. 

The  first  effort  to  model  more  complex  and  cognitive  skills  used  the  UDA  model  without  alterations. 
Work  by  Sabol  et  al.  (1990),  for  example,  has  extended  the  AR1  model  to  tasks  associated  with  mobile 
subscriber  equipment  operator  skills.  These  researchers  used  the  UDA  model  to  make  predictions  of 
the  decay  functions  for  85  skills  needed  to  operate  a  communication  system  called  Mobile  Subscriber 
Equipment  (MSE).  MSE  is  a  radio  telephone  system  and  its  operation  was  considered  to  entail  many 
complex,  highly  cognitive  skills  within  MOS  3  ID  (MSE  Network  Transmission  Operator)  and  MOS 
3 IF  (MSE  Network  Switch  Operator). 

Sabol  et  al.  (1990)  had  six  SMEs  serve  as  raters.  Three  of  the  SMEs  were  experts  on  MOS  3  ID 
procedures  and  three  on  MOS  3  IF  procedures.  Using  task  inventories  generated  during  the 
development  of  the  MSE’s  Soldier’s  Manuals  and  lists  of  skills  generated  during  field  evaluation  of 
MSE,  the  raters  completed  the  ten  question  rating  procedure  of  the  UDA  for  all  85  tasks.  Each  SME 
was  interviewed  separately  and  took  about  two  hours  for  MOS  3  ID  tasks  and  about  three  hours  for 
MOS  3  IF  tasks. 

The  purpose  of  their  study  was  to  predict  retention  of  MSE  operator  skills  and  to  expand  the  UDA  to 
cover  more  complex  skills  than  those  examined  in  its  development.  Sabol  et  al.  (1990)  reported  that  they 
intended  to  obtain  data  with  which  to  compare  predictions  of  the  percentage  of  soldiers  GO  for  each  task 
to  actual  performance  data  of  MSE  operators.  However,  we  were  unable  to  identify  any  publication 
reporting  the  results  of  such  a  comparison.  Although  this  work  only  creates  predicted  (rather  than  actual) 
retention  curves,  it  stands  as  one  effort  to  broaden  the  UDA  model’s  scope  beyond  highly  procedural 
tasks. 

More  recent  work  by  Sabol  and  MacPherson  (2000)7  attempts  an  even  wider  adaptation  of  the  UDA  skill 
retention  model  to  “ digital ”  tasks.  Digital  tasks,  as  defined  by  Sabol  and  MacPherson,  are  tasks 
performed  on  a  computer  such  as  surveillance  tasks  involving  situational  awareness  and  requiring 
communication,  interpreting  information,  problem  solving,  and  making  decisions.  Sabol  and 
MacPherson  (2000)  argue  that  as  the  UDA  model  is  based  on  mainly  procedural  tasks,  it  may  be 
necessary  to  recalibrate  the  model  for  predicting  the  retention  of  computer-based  military  tasks.  If  the 
UDA  is  seen  to  be  an  inadequate  predictor,  then  the  model  will  need  to  be  extended  to  include  a  broader 
range  of  skills.  This  work  appears  to  be  a  first  step  toward  these  goals.  Two  digital  tasks  are  explored  in 
this  work,  including  an  intelligence  gathering  task  (MOS  96B)  and  a  task  related  to  field  artillery  tactical 
data  systems  (MOS  3  ID). 

This  work  began  with  conducting  interviews  with  SMEs  about  these  2  tasks,  and  having  the  SMEs 
generate  predictions  of  the  vulnerability  of  the  task  and  associated  sub-tasks  to  decay.  These 
predictions  were  based  on  an  enhanced  system  of  measurement,  which  incoiporated  new  dimensions 
(see  below)  likely  to  impact  on  the  retention  of  tasks  performed  on  a  computer.  In  addition  to  the 
original  10  UDA  questions,  additional  questions  related  to  the  cognitive  components  of  the  task.  The 
digital  task  retention  activity  matrix  included  the  following  items,  rated  on  a  5 -point  scale,  ranging 
from  the  extent  to  which  the  activity  was  “crucially  involved”  or  “not  at  all  involved”  in  the 
performance  of  the  task.  These  activities  included: 
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•  Following  a  fixed  procedure 

•  Executing  a  continuous  perceptual-motor  action 

•  Monitoring  a  situation  or  system 

•  Adjusting  or  manipulating  equipment 

•  Communicating  information 

•  Receiving,  combining,  interpreting  information 

•  Making  decisions 

•  Solving  problems,  trouble-shooting 

The  following  illustrates  digital  task  retention  questions  completed  by  instructors: 

•  Extent  to  which  situation  awareness  is  a  factor  in  the  completion  of  the  task 

•  How  difficult  it  is  to  maintain  situation  awareness  while  performing  the  task 

•  How  often  a  given  task  is  practiced  during  training 

•  A  self-assessment  of  retention  completed  by  soldiers 

•  Whether  the  task  can  be  performed  in  more  than  one  way 

•  Whether  all  soldiers  are  required  to  learn  all  versions  of  the  task  before  receiving  a  “GO” 

•  Whether  the  equipment  used  has  capabilities  beyond  those  needed  for  the  task 

•  Whether  the  soldier  benefits  from  being  familiar  with  other  capabilities  of  the  equipment. 

These  scores  were  combined  into  a  task  complexity /cognitive  component  score  for  each  task.  SMEs 
also  rated  these  tasks  by  how  likely  each  of  the  tasks  and  sub-tasks  were  to  decay  over  time. 

SME  ratings  of  the  decay  associated  with  each  task  were  regressed  on  both  the  original  1 0  UDA  task 
questions,  and  with  the  expanded  cognitive  and  digital  measures,  with  varying  results.  In  the  first  trial, 
the  original  1 0  UDA  questions  predicted  the  SME  ratings  of  the  task  fairly  well,  and  there  little  added 
benefit  was  found  from  the  additional  “digital”  and  “cognitive”  questions.  In  the  second  trial,  the 
digital  questions  predicted  skill  retention  better  than  the  original  UDA  questions.  Moreover,  these 
tasks  were  rated  by  SME’s  to  be  highly  vulnerable  to  decay.  This  preliminary  report  suggests  that  the 
inclusion  of  measures  specifically  geared  toward  tasks  which  are  primarily  cognitive  may  assist  in  the 
prediction  of  “digital”  tasks,  beyond  what  the  original  UDA  on  its  own  is  able  to  achieve.  However, 
the  work  needs  to  be  examined  in  greater  detail  and  the  nature  of  the  surveillance  in  the  tasks  better 
understood.  There  are  some  apparent  contradictions,  for  example,  in  that  skilled  visual  search  tasks 
such  as  those  involved  in  Air  Traffic  Control,  depending  on  the  manner  of  training,  may  have  minimal 
declines  in  performance  over  periods  as  long  as  a  year  (Swezey  and  Llaneras,  1997,  pp  530-1). 

This  work  is  potentially  important  to  the  prediction  of  collective  skill  retention  from  several 
perspectives.  The  issues  rated  in  the  digital  skill  retention  activity  matrix  include  the  first  attempt 
found  to  address  operationally  the  cognitive  component  of  tasks  in  the  literature  on  prediction  of  skill 
retention.  Furthermore,  understanding  tasks  involving  the  communication  of  information  and  the  need 
for  decision  making,  for  example,  will  be  critical  to  predicting  collective  skill  retention. 

The  existing  literature  (including  the  UDA  model  itself)  argues  that  performing  a  task  often  provides 
feedback  about  the  correctness  of  a  task  step,  facilitating  further  responses.  Spears  (1991b)  notes  that 
failure  to  provide  feedback  to  correct  inappropriate  performance  will  reinforce  the  incorrect  procedure, 
and  affect  retention  and  re-training.  Tasks  without  feedback  are  more  susceptible  to  skill  loss.  For 
collective  skill  retention,  an  increasingly  technological  military  system  may  help  or  hinder  the 
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feedback  process.  Collective  members  need  awareness  of  others  to  dovetail  their  activities. 

Digitisation  may  make  this  less  effective  (witness  concerns  about  digitisation  removing  voice 
communication  and  the  affective  insight  that  can  provide  about  other  team  members.)  Digitisation 
may  also  help  improve  the  situation  through  collation  and  display  of  information  in  ways  that  improve 
comprehension  of  the  progress  of  the  collective  task. 

Within  collectives,  the  challenge  is  that  any  given  collective  member  may  have  access  to  only  one 
small  part  of  the  task,  and  may  or  may  not  know  the  outcome  of  collective  task  performance.  As  such, 
lessened  situation  awareness  in  both  collective  and  increasingly  cognitive  settings  may  also  impact  on 
the  long  term  retention  of  the  related  skills.  Understanding  the  role  of  situation  awareness  will  be 
important  in  future  models  of  collective  and  cognitive  skill  retention. 

One  goal  of  this  review  is  to  consider  the  data  needed  to  advance  models  that  may  be  appropriate  to 
apply  to  collective  skill  retention.  We  would  argue  that  the  recent  work  directed  toward  extending  the 
scope  of  the  UDA  model  to  the  cognitive  domain  is  encouraging  and  necessary  to  enable  prediction 
beyond  simply  procedural  skills. 

On  the  other  hand,  it  could  also  be  argued  that  even  the  original  UDA  model  is  still  in  need  of  strong 
empirical  work  even  for  prediction  of  relatively  simple  procedural  tasks.  Despite  the  promise  of  the 
UDA  in  predicting  skill  retention,  validation  studies  comparing  predicted  retention  with  actual  retention 
is  still  lacking.  Work  to  validate  the  UDA  model  for  procedural  tasks  may  be  more  prudent  than 
expanding  the  model  for  increasingly  elaborate  tasks,  or  in  varying  contexts.  It  is  clear  that  creating 
predictions  about  the  probable  retention  of  tasks  using  the  UDA  models  is  possible.  It  is  now  important 
to  show  that  these  predictions  are  very  closely  related  to  the  actual  performance  on  these  tasks,  as 
measured  by  performance  data.  Also,  the  UDA  approach  as  a  whole  seems  to  lack  a  strong  conceptual 
and  theoretical  framework,  and  that  adding  piecemeal  onto  the  existing  approach  contributes  more  to  this 
atheoretical  and  somewhat  scattershot  nature  of  the  model. 

From  a  broader  perspective,  it  is  unclear  whether  a  UDA  model  that  continues  to  focus  solely  on  task 
characteristics  (however  elaborated)  will  be  adequate  for  modelling  collective  or  cognitive  skill  retention. 
Indeed,  one  of  the  most  common  criticisms  of  the  UDA  model  is  that  although  its  reliance  on  task  factors 
makes  the  UDA  very  ease  to  use,  the  accuracy  with  which  it  predicts  skill  retention  may  be  compromised 
because  it  overlooks  many  other  important  factors.  These  factors  include: 

•  The  degree  of  overlearning  (Hagman  &  Rose,  1983;  Lance  et  al.,  1998; 

Schendel  &  Hagman,  1980,  cited  in  Hagman  &  Rose,  1983), 

•  Opportunity  to  practice  (Blankmeyer,  1998,  Landry  &  Campbell,  1997) 

•  Similarity  of  training  and  performance  environments  (Marmie  &  Healy,  1995). 

•  Individual  aptitude  (Earles  &  Ree,  1992;  Hurlock  &  Montague,  1982). 

We  would  also  argue  that  any  model  that  excludes  features  of  the  collective  (e.g.  organizational 
structure,  number  of  members  within  the  unit  etc),  as  well  as  training,  and  individual  factors  is 
unlikely  to  predict  collective  skill  retention  adequately. 

On  the  other  hand,  using  an  approach  like  the  UDA  to  understand  collective  skill  retention  may  be  an 
important  starting  point.  Having  identified  the  discrete  tasks  associated  with  the  performance  of  a 
given  collective  skill,  training  subjects  to  asymptote  performance,  and  measuring  performance  as  a 
function  of  time  would  provide  a  decay  curve  for  each  task.  With  the  decay  curve  for  multiple  tasks 
known,  it  would  then  be  possible  to  explore  the  relationship  between  various  task  characteristics  and 
the  resulting  curve  for  the  collective  task. 
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6.3  Roth  Model  of  Collective  Skill  Training  and  Retention  (1992) 

Roth  (1992)  represents  the  only  comprehensive  attempt  to  model  collective  skill  retention  found 
during  the  literature  search.  This  model  was  created  to  assist  decision  making  by  unit  planners  about 
collective  training  within  U.S.  Army  units  using  data  currently  available  from  within  the  Standard 
Army  Training  System,  and  to  predict  how  collective  task  performance  would  be  affected  by  intervals 
without  training,  and  by  personnel  turnover  within  the  collective  unit.  Turnover  data  indicates  annual 
turnover  at  the  Battalion  level  as  high  as  60%  to  120%.  Based  on  analyses  of  battalion  training 
schedules,  inter-training  interval  (ITI)  data  indicates  that  most  platoon  collective  tasks  are  practised  as 
infrequently  as  twice  a  year. 

Roth’s  work  was  very  ambitious  both  in  terms  of  the  scope  of  the  factors  incorporated  into  the  model 
and  the  range  unit  and  tasks  studied.  The  work  was  in  four  phases. 

•  The  first  phase  sought  to  establish  a  conceptual  model  to  organize  the  various  factors  seen  to 
influence  the  retention  of  collective  tasks  and  to  understand  how  the  factors  impact  on  small 
military  units  such  as  infantry  squads  and  platoons,  with  30-60  Mission  Essential  Tasks.  Choice 
of  factors  was  based  on  the  effect  on  the  amount  of  information  to  be  learned  and  remembered  to 
accomplish  the  task.  This  phase  also  examined  how  differences  in  the  structure  and  composition 
of  military  units  might  impact  on  predictions  about  collective  skill  retention. 

•  The  second  phase  created  regression  equations  to  predict  retention  of  proficient  collective  task 
performance  over  different  periods  without  training  with  varying  collective  tasks  and  in 
varying  kinds  of  units.  Five  SMEs  (most  with  combat  experience)  provided  estimates  of 
likely  performance  of  collectives  under  various  conditions  of  time  without  training,  and 
considering  varying  levels  of  turnover.  Two  hundred  and  thirty  five  collective  tasks 
performed  by  5  different  types  of  units  were  used  as  a  basis  for  these  predictions.  The 
predictions  formed  using  the  unit-type  and  the  collective  task  categorisation  schemes  were 
then  validated  against  the  regression  equations.  To  a  lesser  extent,  predictions  by  SME’s  were 
compared  against  actual  performance  data  to  validate  the  categorisation. 

•  In  the  third  phase,  a  conceptual  model  of  the  decision  processes  used  by  unit  training  planners 
to  develop  unit  training  was  developed. 

•  In  the  final  phase  of  work,  user  guidance  was  developed  to  support  unit  level  planners  of 
training  including  decision  making  strategies  about  when  and  how  to  provide  collective 
training,  and  cost-effective  training  approaches. 

6.3.1  Theoretical  Bases 

The  first  step  in  Roth’s  work  was  to  define  a  conceptual  model  of  the  process  of  collective  skill 
retention.  In  the  absence  of  other  existing  frameworks  for  understanding  collective  skill  retention, 

Roth  used  the  team  performance  literature  as  a  starting  point.  A  model  from  the  team  performance 
literature  created  by  Bass  (1982)  was  adapted  to  model  collective  skill  retention.  Roth  identified 
unique  factors  likely  to  influence  collective  skill  performance  after  time  without  training  and  used  a 
decision  rule  to  assess  how  these  factors  would  impact  on  skill  retention  within  collectives.  This 
decision  rule  for  inclusion  in  the  model  was: 

“ Does  the  factor  influence  the  amount  that  unit  members  must  learn  and  remember  in 
order  to  accomplish  (a)  collective  task(s)?” 
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This  lead  to  the  conceptual  model,  reproduced  below: 


Characteristics  of  a 
specific  unit  that 
influence  skill  and 
knowledge  maxima 
(members'  aptitude, 
overall  experience, 
experience  as  a 
constituted  team) 


Unit  design  factors  (threat,  missions, 
echelon,  etc.) 


Potentially  expressed 
skills  and  knowledge  of 
unit  members 


Positions  held  by  unit 
members  in  formal 
organizational  structure 
of  the  unit 


Unit  type  characteristics  (number  of 
members,  equipment  used,  formal 
structure,  positions  and  substitutability 
betwsen  them,  leadership) 


Effects  that  limit  the 
unit's  ability  to  express 
maximum  potential  skills 
and  knowledge 
(forgetting,  membership 
change) 


Collective  task  organization  and 
process  modifications 


Characteristics  of  collective  tasks 
performed  by  the  unit  type  and  lower 
echelons 


Individual  tasks  required  to  support 
collective  task  performance 


Performance  conditions  and 
constraints 


Irdvidual  task  performance 


Currently  expressed 
capabilities  (skills  and 
knowledge)  of  unit 
members 


Collective  task 
performance 


Figure  7:  A  conceptual  model  of  factors 
that  influence  collective  task  performance  (Roth,  1992) 

The  output  of  the  model  -  collective  task  performance  -  is  shown  at  the  lower  right  of  the  model.  As 
this  complex  model  above  shows,  Roth  argues  that  there  are  3  broad  classes  or  factors  that  influence 
collective  performance  within  a  collective,  as  a  result  of  time  without  training  and  turnover.  These 
include: 
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•  Unit  Factors  -  associated  with  the  organization,  structure,  and  tasks  performed  by  types  of  teams 
or  units  (e.g.  #  of  members,  formal  organizational  structure). 

•  Collective  Task  Factors  -  associated  with  tasks  performed  by  a  specific  type  or  unit  (e.g. 
number  of  steps,  ability  to  compensate  for  inadequate  performance  etc.) 

•  Individual  Factors  -  associated  with  the  characteristics  of  members  of  a  specific  team  or  unit 
(e.g.  aptitude,  job  experience,  experience  in  performing  as  a  team). 

Although  Roth  specifically  excludes  research  on  team  mental  models  from  the  scope  of  this  work, 
certain  of  the  features  of  Roth’s  model  overlap  with  the  components  proposed  by  research  into  team 
mental  models. 

Roth’s  model  was  seen  to  apply  to  less  structured  groups  as  well  as  with  highly  structured  teams  and 
able  to  be  used  with  all  of  the  tasks  that  collectives  are  likely  to  encounter  including  procedural, 
psychomotor  and  cognitive  tasks.  Each  of  these  broad  classes  of  factors  can  be  further  analysed  in 
terms  of  several  component  factors  (see  below). 

Defining  Unit  Types-  In  the  early  stages  of  work,  Roth  considered  several  different  types  of  small 
military  units,  based  on  the  assumption  that  the  structure  and  composition  of  these  units  would  uniquely 
influence  the  levels  of  skill  retention  within  them.  A  unit  was  defined,  in  part,  by  the  number  of 
personnel  within  this  unit,  as  well  as  by  the  composition  of  the  unit,  in  terms  of  the  proportion  of  unique 
roles  (e.g.  senior  leader,  junior  leader,  and  non-leader)  occupied  by  the  members  of  the  unit.  However,  it 
is  not  clear  if  the  units  studied  were  chosen  to  represent  a  full  range  of  unit  types  with  respect  to  the 
criteria  of  interest.  In  all,  eight  types  of  army  units  were  considered: 

•  Armour  Platoon 

•  Mechanized  Infantry  Platoon 

•  Light  Infantry  Platoon 

•  Light  Infantry  Squad 

•  Mechanized  Infantry  Squad 

•  Tank  crews 

•  MLRS  Firing  Sections 

•  MLRS  Firing  Platoons 

Diagrams  showing  the  composition  of  each  of  these  units  were  also  specified.  For  example,  the 
organization  of  a  Mechanized  Infantry  Squad  was  depicted  in  Figure  8: 
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LEADERS 


VEHICLE  CREW 


Figure  8:  Mechanized  Infantry  Squad  Organization  (Roth,  1992) 

As  a  product  of  examining  these  various  military  units,  Roth  articulated  the  features  of  these  units  that 
might  impact  on  collective  task  performance  as  a  result  of  time  without  training  and  turnover.  These 
factors  are  tabulated  below  showing  the  hypothesized  effect  on  collective  skill  retention  with  respect 
to  time  without  training  and  turnover. 


Proposed  Factor 
and  Definition 

Effect  on  Skill  Loss  as  Result  of 

Time  Without  Training  and  Turnover 

Number  of  unit  members 

Higher  numbers  of  unit  members  necessitate  more  learning.  Therefore,  the  bigger  the 
collective  size,  the  more  skills  fading  that  will  occur  within  a  team.  More  skill  loss  over 
time  in  larger  units,  but  less  loss  related  to  turnover. 

Number  of  sub-teams  in  formal 
unit  structure 

The  more  sub-teams,  the  more  need  to  learn  about  not  only  one's  own  sub-team's 
responsibility,  but  increased  dependencies  with  the  other  sub-teams.  As  such,  higher 
numbers  of  sub-teams  will  create  more  skill  loss  over  time,  and  as  a  result  of  turnover 
than  will  fewer  sub-teams. 

Position  redundancy  in  formal  unit 
structure  -  extent  to  which  unit 
members  can  directly  substitute 
for  other  members 

The  more  that  members  of  a  collective  can  substitute  for  each  other,  the  less  collective 
skill  loss  will  impact  on  them,  as  the  information  and  knowledge  required  by  any 
individual  will  be  less.  More  redundancy  is  associated  with  less  loss  due  to  training 
interval  and  turnover. 

Number  of  equipment  items  used 
by  the  collective 

Each  item  used  requires  knowledge.  The  more  items,  the  more  knowledge  required,  and 
so  the  higher  potential  for  collective  skill  loss  over  time  and  as  the  result  of  turnover. 

Number  of  collective  tasks 
performed  by  a  collective 

With  the  more  tasks  required  of  a  collective,  the  total  pool  of  knowledge  required 
increases.  This  need  for  knowledge  increases  the  possibility  that  collective  skill  loss  will 
occur  within  the  collective  as  a  result  of  time  without  training  and  turnover. 

Number  of  individual  tasks 
performed  by  members  of  a 
collective 

The  account  is  as  above,  but  at  an  individual  level. 

Number  of  leaders  in  formal  unit 
structure 

Leaders  are  posited  to  be  more  likely  to  retain  not  only  their  own  skills,  but  also  those  of 
other  collective  members.  Hence,  having  more  leaders  within  a  collective  will  lead  to  less 
collective  skill  loss  as  result  of  time  without  training  and  turnover. 

Table  12:  Summary  of  Unit  Factors 
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Proposed  Factor 
and  Definition 

Effect  on  Skill  Loss  as  Result  of 

Time  Without  Training  and  Turnover 

Number  of  sub-tasks  and 
standards  in  task  description  - 

When  there  are  more  sub-tasks,  there  is  more  for  unit  members  to  remember  in  order  to 
complete  the  task.  Therefore,  more  knowledge  lost  over  time  for  tasks  with  more  steps 
and  more  loss  related  to  turnover. 

Established  vs.  emergent  nature  of 
task  performance  -  Tasks  which 
are  completed  the  same  way 
under  all  conditions  are 
established,  and  those  which  are 
completed  differently  each  time, 
are  said  to  be  emergent. 

Tasks  with  established  properties  can  be  easily  “cook-booked”,  so  that  simply  learning 
the  steps  to  the  task  may  be  sufficient.  Tasks  which  have  emergent  properties, 
however,  require  more  knowledge  as  adjustments  may  need  to  be  made  for  varying 
contexts.  Highly  established  tasks  require  less  knowledge,  so  are  likely  to  have  less 
skill  loss  over  time,  and  as  the  result  of  turnover. 

Average  number  of  sub-teams 
per  sub-tasks 

The  more  sub-teams  working  on  a  task,  the  more  need  there  is  to  coordinate  activities, 
and  to  learn  about  the  other  sub-teams,  their  membership  and  their  relationship  to  the 
whole  task.  As  such,  a  higher  number  of  sub-teams  predict  less  collective  skill 
retention. 

Number  of  individual  tasks 
required  to  perform  collective  tasks 

Collective  tasks  with  a  higher  number  of  individual  tasks  require  more  information  and 
learning,  so  will  be  retained  less  well  than  collective  tasks  with  fewer  individual  tasks. 

Co-active  vs.  interactive  task 
performance  -  unit  members  do 
similar  (co-active)  or  different 
activities  (interactive)  in  performing 
the  task 

More  interactive  tasks  require  more  knowledge  and  learning  about  other  team  members 
and  the  division  of  responsibility.  These  tasks  will  show  more  skill  loss  as  a  result  of 
time  and  turnover. 

Potential  for  compensating  or 
correcting  errors  during  task 

Tasks  with  more  potential  for  correcting  will  have  less  skill  loss  associated  with  time  and 
turnover,  as  diminished  performance  is  addressed  by  other  team  members. 

Table  13:  Summary  of  Collective  Task  Factors 


The  following  factors  were  identified  but  not  applied  in  the  model. 


Proposed  Factor 
and  Definition 

Effect  on  Skill  Loss  as  Result  of 

Time  Without  Training  and  Turnover 

Aptitude 

High  levels  of  average  individual  aptitude  will  result  in  better  ability  to  learn  and  retain 
knowledge,  and  should  predict  better  collective  skill  retention 

Turnover  rate 

Higher  turnover  will  be  associated  with  lower  collective  skill  retention. 

Experience 

performing  collective  tasks 

Units  in  which  members  have  high  levels  of  experience  may  result  in  overlearning  of 
collective  tasks.  This  may  result  in  higher  levels  of  skill  retention.  This  should  reflect 
the  average  level  of  experience  and  the  highest  level  of  experience  in  the  unit. 

Table  14:  Summary  of  Individual  Factors 

6.3.2  Developing  Regression  Equations  for  Collective  Tasks 

Once  this  categorisation  scheme  for  unit  type  factors  and  collective  task  factors  was  developed, 
regression  equations  were  developed  to  predict  the  likely  effects  of  inter-training  interval  and 
personnel  turnover  on  collective  task  performance  within  specific  kinds  of  units.  Although  the  intent 
was  to  base  these  equations  on  actual  performance  data,  this  was  not  possible,  and  input  from  SMEs 
was  used  instead. 

A  subset  of  the  collective  tasks  performed  by  each  unit  was  used  for  which  the  available  task 
description  provided  sufficient  detail  about  how  the  task  was  to  be  performed.  As  this  work  was  done 
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in  the  U.S.,  the  task  descriptions  came  from  the  training  objectives  for  each  task  and  the  mission 
training  plans  developed  through  the  Army  Training  and  Evaluation  Program. 

Scenarios  defining  varying  levels  of  inter-training  interval  and  turnover  were  then  created  for  each  unit 
type.  For  large  units,  52  scenarios  were  created  for  each,  and  26  scenarios  were  created  for  each  small 
unit.  Within  these  scenarios,  two  inter  training  intervals  were  used:  low  (with  20  to  30%  of  the  unit 
turning  over  each  month)  or  high  (50%  or  more  of  the  unit  turning  over  each  month).  Positions  in 
which  turnover  occurred  were  also  varied  among  senior  leaders,  junior  leaders  or  non-leaders. 

SMEs  were  asked  to  consider  each  scenario,  and  to  estimate  how  well  a  specific  unit  or  collective 
would  perform  on  a  collective  task,  considering  the  unit  turnover  as  well  as  the  time  since  training. 

The  hypothetical  unit  performance  was  rated  as  “T”  (trained),  “P”  (needs  practice),  and  “El” 

(untrained)  for  each  scenario,  and  estimations  were  distributed  in  each  of  these  categories  out  of  100 
total  points  available. 

These  estimations  were  completed  under  three  assumptions:  that  the  unit  began  at  full  strength,  that 
each  unit  member  was  capable  of  completing  each  individual  task,  and  that  the  task  had  performed 
proficiently  immediately  after  training. 

Estimates  were  completed  for  each  of  235  collective  tasks  for  each  unit  types  by  two  SME’s,  working 
independently,  and  five  SME  completed  these  estimates,  in  accordance  with  their  experience.  The 
SME’s  were  retired  Colonels  (2),  Captains  (2)  and  a  Sergeant  (1). 

Estimates  of  inter  rater  reliability  were  conducted  to  ensure  that  the  ratings  of  the  2  SME’s  were 
consistent.  Inter  rater  agreement  correlations  were  conducted  for  ratings  within  each  unit  type,  and 
there  were  shown  to  be  consistently  high,  ranging  between  .66  for  the  Mechanized  Infantry  Squad  and 
.92  for  the  Armor  Platoon. 

The  number  of  points  assigned  to  the  “T”  category  were  averaged  together,  and  these  averages  were  used 
as  criterion  data  for  developing  the  regression  equation.  The  predictor  variables  were: 

•  Number  of  months  since  training 

•  %  turnover  of  senior  leaders 

•  %  turnover  of  junior  leaders 

•  %  turnover  of  non-leaders 

The  regression  equation  form  used  in  this  research  was: 

Predicted  Y  =  constant  +  Bm  +  BslxSl  +  B JLx JL  +  BNLxNL 

This  regression  equation  describes  the  probability  that  a  unit  will  be  trained  if  it  performs  the  task  under 
the  specified  conditions  of  inter  training  interval  and  personnel  turnover.  The  B  terms  are  the  weights  or 
coefficients  for  inter  training  interval,  senior  leader  turnover,  junior  leader  turnover,  and  non-leader 
turnover.  The  x  terms  are  the  actual  values  of  inter  training  interval  (in  months)  and  turnover. 

A  stepwise  regression  procedure  was  used  to  develop  the  regression  equations.8  Predictor  variables 
were  included  if  they  increased  the  coefficient  of  multiple  correlation  between  the  set  of  predictors  and 
the  criterion  variable.  Estimates  were  completed  for  235  tasks,  and  regression  equations  were  created 


8  Using  stepwise  regression  is  obviously  problematic.  Rather  than  forming  predictions  about  which  predictors 
were  likely  to  exert  influence  and  testing  this,  stepwise  regression  amounts  to  throwing  all  the  predictors  into  the 
mix.  Most  critically,  it  is  not  clear  that  the  results  are  necessarily  helpful  as  a  basis  for  theory  development. 
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for  each  of  the  estimated  tasks  paired  with  the  appropriate  unit.  Linear  regression  equations  were 
adequate  for  describing  the  variance  between  SME  performance  estimates  and  the  predictor  variables, 
with  the  Average  Multiple  r  ranging  between  .96  and  .99  for  the  5  units. 

The  average  predicted  skill  loss  due  to  inter  training  interval  ranged  from  1.89%  for  Light  Infantry 
Squads  to  about  4.07%  for  Mechanized  Infantry  Squads.  Over  all  units,  the  average  performance 
decrement  due  to  inter  training  interval,  as  predicted  by  the  SME  estimates  was  about  3%  per  month. 

Understanding  the  average  effect  of  turnover  is  more  difficult,  as  the  weights  provided  are  based  on 
varying  numbers  associated  with  leader  turnover.  Using  an  estimate  of  only  1%  per  month  in  all 
categories,  the  average  effect  of  leader  turnover  within  varying  units  ranged  from  -.35%  to  -1.69%  for 
senior  leaders,  from  -.077%  to  -.41%  for  junior  leaders  and  from  -.13%  to  -.23%  for  non-leaders,  with 
the  effect  of  turnover  predicted  to  be  somewhat  more  influential  within  a  mechanized  infantry  platoon 
than  within  other  units. 

So,  after  this  intensive  process,  regression  equations  representing  the  predicted  amount  of  skill  loss 
given  differing  amounts  of  time  without  training  and  turnover  were  created  for  each  collective  task 
within  each  unit. 

6.3.3  Applying  and  Validating  the  Categorization  Schemes 

The  next  step  in  developing  the  model  was  to  show  that  the  factors  identified  were  related  to  the 
retention  predictions  derived  from  the  regression  equations.  SMEs  first  rated  each  of  the  unit  types  on 
the  unit-type  factors,  and  then  rated  the  collective  tasks  on  the  collective  task  factors.  Ratings  on  the 
unit  type  factors  were  completed  by  counting  the  number  of  sub-tasks  in  the  task  and  the  number  of 
equipment  items  used  by  the  unit  type. 

For  the  collective  factors,  the  process  was  much  more  complicated,  and  required  the  development  of 
several  rating  scales.  To  validate  the  model  as  a  whole,  Roth  first  attempted  to  validate  classification 
of  unit  types  and  collective  factors  on  the  amount  of  collective  skill  loss. 

A  single  score  was  created  to  represent  the  overall  effect  of  the  unit  type  factors  combined  (UTEP 
score).  For  this,  the  unit  type  raw  scores  for  each  of  the  unit  type  factors  were  converted  into  “high” 
and  “low”  effects  indexes  for  both  the  inter  training  interval  and  turnover,  based  on  the  position  of  each 
score  relative  to  the  median  on  each  of  these  2  dimensions.  If  a  given  unit  type  factor  was  rated  above 
the  median  in  terms  of  its  hypothesized  effect  on  both  ITI  and  turnover,  it  would  yield  a  score  of  2.  The 
scores  for  each  of  the  6  factors  were  then  added  together  to  yield  a  total  score  ranging  from  0  to  12. 

For  units  with  higher  UTEP  scores,  retaining  collective  performance  would  be  more  difficult. 

A  similar  process  was  used  to  develop  a  collective  task  effect  prediction  (CTEP  score).  SME’s  rated 
each  of  the  tasks,  and  created  scores  for  each  task  within  each  kind  of  unit.  CTEP  scores  were  divided 
into  “high”  and  “low”  effect  categories,  based  on  a  cut-point  approximately  equal  to  the  median  for  each 
task  on  the  ITI  and  turnover  dimensions.  These  “high”  and  “low”  designations  were  then  combined  to 
give  an  index  of  the  predicted  effects  of  ITI  and  turnover  on  the  specified  task. 

Higher  CTEP  scores  indicate  tasks  with  more  collective  skill  loss  over  time  and  from  turnover. 

Individual  factor  scores  from  the  unit  type  rating  and  the  collective  task  categorisation  ratings  and  the 
UTEP  and  CTEP  scores  were  validated  against  the  SME  estimates  of  collective  task  performance 
change.  The  goal  was  to  examine  the  extent  to  which  the  predictions  implicit  in  the  UTEP  and  CTEP 
scores  (as  well  as  in  the  individual  factor  ratings)  would  match  with  the  predictions  from  the 
regression  equations. 
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a)  Unit-Type  Factors  and  UTEP  Score  Validation  Against  SME  Data 

The  first  step  was  to  work  to  validate  individual  factors  related  to  unit  type,  as  well  as  the  global 
UTEP  score  created  to  represent  unit  type.  A  strong  relationship  between  the  prediction  of  retention  in 
the  SME  estimates,  and  the  unit  type  factors  would  be  evidence  for  the  validity  of  the  classification 
scheme.  An  average  beta  weight  was  calculated  for  each  of  the  4  factors  (ITI,  turnover  x3)  across  all 
collective  tasks  for  each  unit  type,  in  order  to  establish  the  strength  of  each  of  the  4  factors  for  each 
unit  type. 

These  average  beta  weights  were  then  correlated  with  the  raw  data  values  for  each  unit-type  factor  and 
with  the  UTEP  score  assigned  to  each  of  5  unit  types. 


Unit  Type 

Classification  Factors 

Characteristics  of  Regression  Equations 

Based  on  SME  Estimates 

UTEP  Score 

Average  beta  weight  for  effect  of  ITI 

Number  of  Members 

Average  beta  weight  for  effect  of  Non-Leader  Turnover 

Number  of  Sub-Teams 

Position  Redundancy  Measure 

#  of  Equipment  Items  per  Member 

#  of  Collective  Tasks 

#  of  Individual  Tasks 

#  of  Leaders 

Correlated 

against 

Average  beta  weight  for  effect  of  Senior  Leader  Turnover 
Average  beta  weight  for  effect  of  Junior  Leader  Turnover 

Table  15:  Analyses  to  Validate  Unit  Type  Categorisation  Scheme  (Roth,  1992) 

This  led  to  a  total  of  28  possible  correlations,  of  which  10  were  significant  at  p  <  .05,  or  close.  The 
table  of  significant  correlations  is  provided  below. 


Correlations  Between: 

r 

N 

P 

UTEP 

Senior  Leader  Turnover 

-.82 

5 

.044 

#  of  Members 

Junior  Leader  Turnover 

-.91 

4 

.045 

Position  Redundancy 

Junior  Leader  Turnover 

-.92 

4 

.040 

#  of  sub-teams 

Junior  Leader  Turnover 

-.92 

4 

.040 

#  of  collective  tasks 

Junior  Leader  Turnover 

-.98 

4 

.011 

#  of  leaders 

Junior  Leader  Turnover 

-.90 

4 

.051 

#  of  equipment  items/member 

Senior  Leader 

-.76 

5 

.064 

#  of  individual  tasks 

Senior  Leader 

-.83 

5 

.042 

#  of  equipment  items/member 

Inter  training  Interval 

-.86 

5 

.032 

#  of  individual  tasks 

Inter  training  Interval 

-.86 

5 

.032 

Table  16:  Significant  Correlations  Between  Unit  Type  Categorization  Variables  and 
Average  B  Weights  in  Regression  Equations  for  Unit  Types  (Roth,  1992). 

As  this  table  shows,  the  correlation  coefficient  pattern  is  consistent,  as  increases  in  unit-type  factors 
(e.g.  larger  teams)  are  associated  with  predictions  of  lower  collective  task  performance  in  the 
equations  predicting  skill  loss  for  a  given  unit  type. 

Moreover,  all  of  the  unit-type  factors  are  significantly  correlated  with  at  least  one  of  the  equation 
terms.  Number  of  equipment  items  and  number  of  individual  tasks  correlate  with  both  ITA  and  senior 
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leader  turnover.  The  other  factors  correlate  with  just  junior  leader  turnover,  and  no  factors  correlated 
with  non-leader  turnover. 

Importantly,  the  overall  UTEP  score  (meant  to  be  a  global  indicator  of  the  ITI  and  turnover  on  unit 
type)  only  correlated  with  senior  leader  turnover,  suggesting  that  the  UTEP  score  is  a  poor  predictor  of 
retention.  This  suggests,  as  one  might  expect,  that  leader  stability  is  an  important  issue  within  units,  as 
a  predictor  of  collective  skill  retention.  Overall,  Roth  argues  that  there  is  some  evidence  that  the 
individual  unit  factors  are  related  to  collective  skill  retention,  but  little  support  for  UTEP  as  a  tool. 
However,  the  possible  bias  of  SMEs  (as  leaders  themselves),  towards  overestimation  of  the 
importance  of  leadership  positions  needs  to  be  considered. 

b)  Collective  Task  Factors  and  CTEP  Validation  Against  SME  Data 

The  next  step  was  to  validate  the  collective  task  factors  seen  to  influence  collective  skill  retention. 
Again,  an  average  beta  weight  was  calculated  for  each  of  the  4  factors  (ITI,  turnover  x3)  across  all 
collective  tasks  for  each  unit  type,  in  order  to  establish  the  strength  of  each  of  the  4  factors  for  each 
collective  task. 


For  Each  Unit  Type  Separately, 

Collective  Task 

Classification  Factors 

correlated 

against 

Components  of 

Regression  Equations 

Based  on  SME  Estimates 

CTEP  Score 

B  Weight  for  Effect  of  ITI 

B  Weight  for  Effect  of  Non-Leader  T urnover 

B  Weight  for  Effect  of  Junior  Leader  Turnover 

B  Weight  for  Effect  of  Senior  Leader  Turnover 

Number  of  Sub-tasks  &  Standards 

Established  versus  Emergent  Rating 

Number  of  Individual  Tasks 

Coactive  versus  Interactive  Rating 

Rating  of  Potential  for  Correction  or  Compensation  in 
Task  Performance 

Table  17:  Summary  of  Analysis  to  Validate  Collective  Task  Categorisation  Scheme 

These  average  beta  weights  were  then  correlated  with  the  raw  data  values  for  each  collective  task 
factor  for  each  unit  type  and  with  the  CTEP  score  for  each  collective  task.  Out  of  many  possible 
correlations,  20  were  statistically  significant  and  are  shown  in  the  table  below. 
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Unit  Type 

Correlation  Between 

r 

N 

P 

Armor 

Platoon 

Established/Emergent  Rating 

Senior  Leader  Turnover 

-.4754 

64 

<.001 

Established/  Emergent  Rating 

ITI 

-.3520 

64 

<.01 

Number  of  Individual  Tasks 

Non-leader  Turnover 

-.2772 

58 

<.02 

Number  of  Individual  Tasks 

Senior  Leader  Turnover 

-.2350 

61 

<.03 

Coactive/Interactive  Rating 

Non-leader  Turnover 

-.2520 

61 

.03 

Coactive/Interactive  Rating 

Senior  Leader  Turnover 

-.3097 

64 

<.01 

Coactive/Interactive  Rating 

ITI 

-.3016 

64 

<.01 

Mechanized 

Infantry 

Platoon 

Number  of  Individual  Tasks 

Junior  Leader  Turnover 

-.3509 

58 

<.01 

Number  of  Individual  Tasks 

Senior  Leader  Turnover 

-.2201 

58 

<.05 

Number  of  Individual  Tasks 

ITI 

-.3325 

58 

,.01 

Light  Infantry 
Platoon 

Number  of  Individual  Tasks 

Non-leader  Turnover 

-.3218 

44 

<.03 

Number  of  Individual  Tasks 

ITI 

-.3497 

44 

<.01 

Light 

Infantry 

Squad 

Number  of  Sub-teams  per  Sub-task 

Non-leader  Turnover 

.6426 

33 

<.001 

Number  of  Sub-teams  per  Sub-task 

Junior  Leader  Turnover 

-.3065 

36 

<.04 

Number  of  Sub-teams  per  Sub-task 

ITI 

-.5186 

38 

<.001 

Number  of  Individual  Tasks 

Non-leader  Turnover 

-.4213 

33 

<.01 

Number  of  Individual  Tasks 

ITI 

-.4052 

38 

<.001 

Coactive/Interactive  Rating 

Junior  Leader  Turnover 

.3000 

36 

<.04 

Coactive/Interactive  Rating 

ITI 

.2774 

38 

<.05 

Potential  for  Compensation  or  Correction 

Non-leader  Turnover 

.3266 

33 

<.04 

Table  18:  Significant  Correlations:  Collective  Task  Factors  and  Regression  Components. 

Correlations  were  mostly  low  indicating  considerable  variability,  although  results  did  show  the 
expected  pattern  of  negative  correlations  between  the  beta  weights  for  the  regression  equations  and 
increases  in  the  collective  task  factors,  even  in  varying  unit  types.  Results  are  more  consistent  at  the 
platoon  level  than  for  the  smaller  squad  where  there  were  several  inconsistencies  (e.g.  positive 
correlations)  and  a  lack  of  significant  correlations.  Roth’s  explanation  is  that  this  work  may  best 
describe  collective  skill  retention  at  the  Platoon  level 

The  number  of  individual  tasks  performed  by  personnel  within  varying  units,  correlates  most  often 
with  inter  training  interval,  as  well  as  with  Senior  Leader  turnover.  Ratings  of  task  co-activity  and 
interactivity  are  also  frequently  correlated  with  the  regression  equation  weights.  Again,  however, 
there  is  no  evidence  that  the  combined  measure  of  the  collective  task  effect  is  at  all  correlated  with 
higher  collective  skill  loss  (as  indicated  by  the  regression  weights).  Although  any  large  battery  of 
correlations  is  likely  to  show  at  least  some  statistically  significant  relationships,  there  is  some  evidence 
that  the  collective  task  factors  as  specified  are  correlated  with  the  subject  matter  estimates  of  skill  loss. 
However,  the  correlations  are,  for  the  most  part,  small  and  their  practical  significance  should  not  be 
over-interpreted. 

In  conclusion,  although  efforts  to  validate  these  factors  are  brave,  the  results  of  these  efforts  are 
questionable  at  best.  Consequently,  it  is  very  difficult  to  assess  the  validity  of  the  proposed  framework, 
and  the  categorisation  approach  merits  further  refinement  and  more  effort  at  validation. 

6.3.4  Empirical  Evaluation 

After  this  validation  of  the  categorisation  schemes,  Roth  worked  to  assess  the  empirical  strength  of  the 
model  by  comparing  the  SME  predictions  of  collective  skill  retention  against  actual  performance  data. 

In  the  first  test  of  the  collective  task  categorisation  schemes  with  real  data,  predicted  scores  on  the 
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individual  collective  tasks  and  the  overall  CTEP  scores  were  validated  against  actual  task  performance 
data  for  Light  Infantry  Platoon  tasks. 

c)  Collective  Task  Factors  and  CTEP  Validation  Against  Actual  Performance  Data 

Ideal  validation  of  collective  task  factors  would  be  to  compare  predicted  scores  on  collective  tasks 
(from  SME’s  estimates  in  the  regression  equations),  and  the  actual  performance  of  a  unit  on  those  tasks. 
Performance  evaluation  data  for  Light  Infantry  Platoons  had  been  preserved  over  the  course  of  12 
training  rotations  at  a  training  centre.9  This  enabled  “ predicted  to  actual ”  comparisons  for  selected 
platoon  tasks. 

Two  platoon  task  performance  measures  were  available  from  the  training  centre. 

•  The  first  was  the  ratio  of  task  steps  rated  “GO”  over  all  the  steps  rated  on  each  of  1230 
tasks  completed  by  a  Light  Infantry  Platoon  unit  task,  ranging  from  0  to  1  over  a  period  of 
12  rotations  at  the  JRTC.  These  tasks  matched  the  tasks  used  in  the  creation  of  the 
regression  equations. 

•  The  second  performance  measure  was  based  on  the  number  of  sub-tasks  scored  “GO”, 
using  859  tasks  matching  the  Light  Infantry  Platoon  tasks. 

Two  different  analyses  were  completed. 

In  the  first,  “high”  and  “low”  declarations  were  given  to  effects  ratings  on  group  variables  with  the 
expectation  that  higher  CTEP  scores  and  high  effect  declarations  would  be  related  to  lower  levels  of 
performance.  Median  tests  were  conducted  to  assess  the  relationships  between  these  variables.  Results 
showed  that  only  three  factors  that  correlated  significantly  with  the  performance  data:  the  number  of 
individual  tasks,  co-active/interactive  ratings  and  potential  for  compensation  or  correction. 

This  suggests  that  the  number  of  sub-tasks  and  standards  was  not  a  good  predictor  of  collective  task 
performance,  nor  was  the  established/emergent  rating.  There  was  a  statistically  significant  effect  for 
the  number  of  sub-teams  per  sub-task,  but  this  effect  was  in  the  opposite  direction  from  what  was 
expected. 

The  second  analysis  looked  at  the  correlation  between  the  CTEP  score  and  actual  performance,  using 
the  prediction  equations.  The  ITI  was  set  to  3  months  and  the  turnover  (3  kinds)  was  set  to  33%  per 
month.  These  predictions  were  then  correlated  with  the  actual  performance.  The  resulting  correlation 
between  the  combined  CTEP  score  and  performance  score  was  statistically  significant  but  very  low, 
ranging  between  minus  0.12  and  minus  0.23.  This  suggests  that  the  CTEP  score  is  a  poor  predictor  of 
collective  task  performance. 

Additional  analyses  were  conducted  to  compare  the  correlation  between  performance  estimates  made  by 
SME’s  and  the  actual  performance  data.  These  analyses  can  be  seen  as  an  indicator  of  the  validity  of  the 
SME  estimates.  Regression  equations  for  the  collective  tasks  in  the  JRTC  data  were  applied  using  a 
common  scenario  of  ITI  (3  months)  and  turnover  (33%).  Results  showed  a  consistent  but  very  modest 
correlations  between  SME  estimates  and  actual  task  performance  scores  ranging  between  r  =  0. 14  and  r  = 
0.36.  The  statistical  significance  of  the  positive  correlations  suggests  that  when  SME’s  estimated  the 
retention  of  a  given  task  to  be  low,  actual  performance  of  the  task  was  also  low.  However,  the  low 
correlation  shows  that  the  SME  estimates  have  only  a  very  weak  relationship  with  actual  task 
performance. 


9  The  time  interval  reported  between  sessions,  and  the  total  interval  over  the  12  sessions  was  not  reported. 
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Correlations  comparing  the  actual  JRTP  performance  data  and  the  CTEP  scores  were  also  conducted. 
These  analyses  suggest  no  relationship  between  the  CTEP  scores  and  the  actual  performance  scores,  r 
=  0.031. 

Summary  Overview  -  These  validation  results  provide  very  limited  evidence  that  the  collective  task 
factors  as  combined  in  this  model  are  able  to  predict  collective  skill  retention.  There  is  some  evidence 
that  three  of  the  six  factors  are  correlated  with  predictions  of  skill  retention  but,  overall  the  results  of 
this  empirical  evaluation  are  far  from  convincing,  and  suggest  that  considerable  work  is  still  required 
to  understand  how,  if  at  all,  the  factors  proposed  influence  collective  skill  retention. 

Although  this  work  did  use  actual  performance  data  for  validation,  it  is  difficult  to  assess  the  failure  of 
the  model  to  achieve  clear-cut  results.  The  interval  elapsed  between  initial  training  and  the  actual 
performance  measures  taken  is  not  specified.  With  so  many  tasks,  factors,  and  measures,  combined 
with  puzzling  analytic  procedures,  it  is  difficult  to  tease  out  underlying  issues  and  suggest  likely 
improvements,  except  to  say  that  a  less  ambitious  and  more  surgical  approach  comparing,  perhaps  two 
contrasting  unit  types,  and  two  contrasting  tasks,  in  greater  depth  and  using  customized  rather  than 
found  empirical  data  would  have  been  more  productive  approach  in  the  long  run.  Although  our  search 
was  unable  to  find  any  other  research  that  uses  this  model  in  any  way,  such  work  would  be  helpful. 

6.3.5  Applicability  and  Practicality 

Although  the  empirical  value  of  the  model  is  questionable,  this  work  does  clearly  lay  out  a  framework 
for  factors  likely  to  impact  on  collective  skill  retention.  The  discussion  of  potentially  influential 
collective  task  factors  and  the  unit  type  factors  is  very  helpful,  and  represents  the  best  discussion  of  the 
factors  likely  to  impact  on  collective  skill  retention  found  in  our  review. 

From  a  practical  perspective,  the  Roth  research  shows  a  lead  in  one  approach  to  predicting  the 
retention  of  collective  skills.  Although  the  rating  process  is  likely  to  be  very  arduous,  the  sheer 
number  of  tasks  and  unit  types  rated  for  this  research  present  evidence  that  it  is  possible  for  SMEs  to 
use  the  proposed  measures  to  make  ratings  of  collective  task  retention. 

The  potential  for  using  a  similar  approach  in  future  work  remains  dependent  on  the  ability  to 
meaningfully  represent  the  tasks  performed  within  varying  contexts.  This  requires  having  and/or 
developing  detailed  task  analyses  of  each  task  and  sub-task.  In  describing  his  model,  Roth  made  the 
assumption  that,  within  military  contexts,  many  of  these  task  conditions  and  standards  already  exist. 
This  may  or  may  not  be  the  case  in  the  military  contexts  touched  by  this  review.  As  such,  a  critical  part 
of  preparing  to  use  this  model  is  to  do  the  preparatory  work  of  task  and  sub-task  analysis. 

In  terms  of  the  type  of  skills  to  which  this  model  be  applied,  the  235  tasks  used  in  this  work  seem  to 
represent  a  broad  spectrum  of  skills,  ranging  from  perceptual-motor  skills  to  more  complex  and 
primarily  cognitive  skills.  In  general,  however,  the  focus  appears  to  be  slightly  more  on  execution 
skills  (e.g.  clear  a  wood  line)  than  on  decision-making  skills  (e.g.  prepare  for  tactical  operations).  The 
unit  type  factors  identified  also  theoretically  allow  this  model  to  be  used  in  varying  contexts,  ranging 
from  very  small  teams  (e.g.  tank  gunnery  crews)  to  higher  levels  of  organization  (e.g.  command  and 
control  teams).  The  focus  within  this  work,  however,  is  at  the  platoon  level  and  below.  It  is  unclear 
how  well  more  complex  organizational  structures  and  C2  tasks  could  be  captured  using  the  collective 
factors  identified  within  this  work. 

In  terms  of  the  data  needed  to  advance  the  model,  there  is  clear  potential  for  using  the  categorisation 
schemes  proposed  in  order  to  predict  collective  skill  retention,  but  these  need  validation.  These 
categorisation  schemes  are  plausible  and  have,  at  the  very  least,  face  validity.  Moreover,  using  the 
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tools  provided,  it  would  also  be  possible  to  perform  another  validation  of  the  categorisation  scheme 
using  real  performance  data,  rather  than  correlating  subjective  estimates  with  more  subjective 
estimates. 

Surprisingly,  however,  this  work  does  not  appear  to  have  been  cited  in  associated  research.  In  fact,  it 
is  unclear  why  this  article’s  distribution  and  dissemination  within  the  skill  retention  literature  has  not 
been  more  widespread,  and  why  its  contribution  has  not  been  more  influential.  It  seems  possible  that 
the  subjective  basis  of  the  model  and  the  analytic  muddle  within  the  results  presented  may  be  at  least 
partly  responsible  for  its  low  profile  within  the  skill  retention  research. 

Despite  the  overly  ambitious  goals  and  the  analytic  shortcomings  of  this  work,  however,  we  would  argue 
that  this  article  makes  two  important  contributions.  First,  and  to  its  credit,  the  work  addresses  the 
prediction  of  collective  skill  retention  with  a  model  building  approach  that  appears  to  be  all  but 
overlooked  in  the  available  literature.  The  creation  of  a  conceptual  model  for  collective  skill  retention  is 
a  significant  contribution,  and  Roth’s  work  represents  the  only  attempt  found. 

Secondly,  Roth’s  provides  a  good  analysis  of  the  factors  likely  to  impact  on  collective  skill  retention. 
Many  of  the  task  factors  cited  are  identified  in  other  theory  and  research,  but  Roth  adds  specification 
of  these  factors,  combined  with  articulation  of  how  they  are  likely  to  be  impacted  by  retention  interval 
and  by  turnover  within  the  collective. 

6.4  Comparing  Existing  Models  to  the  Categorisation  Framework 

In  this  section,  we  consider  each  of  the  three  models  reviewed  in  terms  of  our  categorisation  framework 
of  factors  to  be  considered  in  future  models  of  collective  skill  retention.  The  categorisation  framework 
is  intended  as  a  guide  for  thinking  about  collective  and  cognitive  skill  retention,  rather  than  a  formal 
specification  of  all  the  features  to  be  considered.  In  fact,  in  the  next  chapter,  we  argue  against  including 
all  of  the  defined  factors  (or  even  sets  of  factors)  in  future  modelling  attempts.  In  many  ways,  the  grand 
scale  is  one  of  the  main  drawbacks  of  the  Roth  model.  Rather,  we  propose  the  framework  as  a  tool  for 
considering  the  factors  likely  to  impact  on  collective  and  cognitive  skill  retention  in  a  broad  range  of 
settings.  Actually  beginning  to  understand  such  a  phenomenon  will  require  focusing  concentrated 
efforts  on  a  limited  number  of  factors  within  a  specific  setting. 

The  categorization  table  shows  that  Nichol’s  model  is  the  most  rudimentary,  as  it  focuses  only  on  task 
difficulty,  importance  and  frequency.  Once  the  other  models  are  incorporated,  there  is  no  important 
contribution  of  the  Nichol’s  model  that  is  not  addressed  in  the  UDA  and/or  the  Roth  model. 

The  original  UDA  model  with  some  adaptation  has  the  ability  to  address  many  of  the  collective  task 
factors  identified  in  this  review.  In  addition,  more  recent  research  efforts  extending  the  UDA  model  to 
cognitive  (digital)  skills  extend  the  scope  of  the  UDA  model  to  address  more  cognitive  skills  such  as 
situation  awareness  and  problem  solving.  We  have  included  recent  research  efforts  as  part  of  the 
potential  predictive  ability  of  the  UDA  model,  even  though  the  original  version  of  the  model  did  not 
incorporate  these  factors. 

As  the  categorization  framework  also  shows,  the  UDA  model  continues  to  focus  on  task  factors  (albeit 
increasingly  elaborated  task  factors),  ignoring  several  other  critical  influences  on  collective  skill 
retention,  such  as  training  factors,  individual  learner  factors,  and  the  features  of  a  collective’s 
structure,  organization,  and  composition. 

Despite  being  at  the  relatively  early  stages  of  development,  the  Roth  model  provides  the  most 
comprehensive  account  of  collective  skill  retention  found  in  the  existing  literature.  Roth  addresses 
many  of  the  factors  identified  in  this  review.  Identification  of  factors  related  to  collective  units  is 
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arguably  the  most  important  contribution  of  this  work.  The  Roth  model  considers  the  number  of 
collective  members,  as  well  as  the  number  of  leaders  as  potential  predictors  of  skill  retention.  The 
argument  that  skill  retention  within  collectives  will  be  affected  by  the  structure,  organization,  and 
composition  of  the  collective  is  an  important  contribution.  Personnel  turnover  within  the  collective  is 
another  unique  contribution  of  the  Roth  model,  and  an  important  factor  worthy  of  serious 
consideration  for  future  models,  particularly  within  the  military  domain.  Turbulence,  or  changes  in 
the  roles  and  responsibilities  within  collectives  is  also  a  factor  likely  to  impact  on  the  retention  of 
collective  performance,  though  the  direction  of  this  effect  will  depend  on  the  nature  of  the  cross¬ 
training  achieved.  Moreover,  the  ability  to  move  the  task  factors  identified  in  previous  models  of 
individual  skill  retention  into  collective  settings,  and  to  articulate  these  factors  at  a  collective  level  is 
also  evident  within  the  categorisation  framework.  For  the  future,  Roth’s  model  could  be  used  as  a 
spring  board. 


Yinmansystems®  Incorporated 


Collective  and  Cognitive  Skill  Retention 


Page  81 


Models  reviewed1 

Factors 

N 

ARM 

ARI-2 

R 

Collective 

asks 

md 

sub-tasks 

1 .  Number  of  collective  tasks  assigned  to  the  unit 

■/ 

2.  Number  steps  in  individual  sub-tasks 

✓ 

3.  Availability  of  interstep  cueing  within  sub-tasks 

✓ 

■/ 

4.  Predictability  of  sub-task  procedures  &  conditions  of  performance. 

5.  Number  of  discrete  skills  required  by  the  collective  task 

■/ 

6.  Number  of  sub-units  needed  to  complete  sub-tasks 

■/ 

7.  Interchangeability  among  collective  sub-tasks 

✓ 

8.  Interdependence  among  collective  sub-tasks 

9.  Potential  for  error  correction  and  resource  re-deployment. 

✓ 

10.  Need  for  information  exchange  among  collective  members 

1 1 .  Cognitive  complexity  of  collective  sub-tasks. 

✓ 

12.  Criticality  of  sub-task 

✓ 

✓ 

■/ 

Collective 

structure 

1 .  Structure  of  the  collective  (sub-units  and  levels  of  organization). 

2.  Number  of  people  in  the  collective. 

■/ 

3.  Leadership  structure 

4.  Potential  for  role  substitution 

✓ 

Collective 

nember-shif 

1 .  Shared  mental  models  of  team  members  and  team  tasks. 

2.  External  turnover  and  internal  turbulence 

3.  Social  environment  in  the  collective. 

ndividual 

inferences 

1 .  Aptitudes  of  collective  members 

2.  Diversity  of  experience  among  collective  members 

3.  Experience  in  collective  settings. 

Training 

eceived 

1 .  Level  of  learning  after  initial  training. 

•/ 

V 

2.  Time  since  training:  retention  interval 

V 

V 

✓ 

✓ 

3.  Match  between  the  training  context  and  the  retention  context. 

4.  Training  received  as  a  collective 

✓ 

5.  Training  strategies  used  to  achieve  collective  performance. 

6.  Acquisition  phase  of  collective  task 

? 

Dther 

1 .  Number  of  items  of  equipment  and  their  complexity 

■/ 

Table  19:  Summary  of  factors  likely  to  influence  collective  skill  retention. 

^Models:  N  =  Nichols;  ARI  =  Army  Research  Institute;  R  =  Roth.  2S  =  present  in  some  form.  ?  =  indirectly  present.) 

None  of  the  models  adequately  address  training  issues  which  are  likely  to  impact  on  collective  skill 
retention  and  although  the  importance  individual  differences  in  aptitude  are  noted  in  all  3  models,  only 
the  Roth  model  explicitly  articulates  the  importance  of  these  factors,  even  though  those  factors  were  not 
incorporated  into  the  model. 

As  a  whole,  these  models  represent  a  good  starting  point  to  begin  to  understand  how  to  predict  the 
retention  of  skills  within  collectives  over  time.  However,  it  is  also  clear  that  considerable  effort  and 
research  will  be  necessary  to  make  progress  in  this  area.  The  next  section  presents  some 
recommendations  for  beginning  this  process. 
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A  Way  Ahead  for  Predicting  Retention  of 
Collective  Performance 


This  section  of  the  report  begins  by  exploring  broad  considerations  for  development  of  an  approach 
for  predicting  collective  skill  retention.  The  second  part  includes  recommendations  for  areas  of 
research  and  theory  that  would  be  beneficial  to  explore  in  more  detail.  The  illustration  below  outlines 
the  factors  discussed  in  earlier  sections  in  relation  to  acquisition,  retention  and  re-acquisition. 


INITIAL  TEST 

Stage  of  learning 
Speed  /  accuracy 


COLLECTIVE  STRUCTURE 
-Horizontal  divisions 
-Vertical  levels 

-  Role  substitution 

-  Redundancy 


COLLECTIVE  MEMBERS 
Shared  mental  models 
Social  environment 
Turnover 
Turbulence 


RETEST  CONDITIONS 

Similarity  initial  test 

-  context 

-  members 

-  sub-tasks 


ACQUISITION 
Training  strategies 
Aptitude  of  members 
Experience  of  members 
Task  complexity 
Team  mental  model 


COLLECTIVE  TASK  AND  SUB-TASKS 

-  Inter-dependence  of  sub-tasks 

-  Criticality  of  sub-tasks 

-  Interaction  among  sub-tasks 

-  Information  exchange  among  members 

-  Diversity  of  skills  required 

-  Sub-Task  inter-changeability 

-  Predictability  of  procedures 

-  Potential  for  error  detection  and  correction 


RETENTION  INTERVAL 
Length 

Other  training 
Other  collective  tasks 
Cross  training  effect 
Transfer  of  training  effects 


Figure  9:  Factors  likely  to  influence  retention  of  collective  performance 


7.1  A  way  ahead 

Before  discussing  any  way  ahead,  some  broad  observations  are  appropriate. 

The  whole  area  of  interest  (i.e.  training  acquisition,  retention  and  re-acquisition  at  the  individual  and 
group  level)  is  replete  with  confusing  and  inconsistent  results,  employing  often  very  simple  or 
unrealistic  tasks,  and  using  subjects  with  inappropriate  or  unknown  experience,  aptitudes  and/or  skills. 
Methods  are  more  often  laboratory  than  field  based,  and  inconsistently  validated.  Indices  of  retention 
and  measures  of  perfonnance  vary  widely,  and  seldom  balance  speed  and  accuracy.  Individual 
training  issues  such  as  learning,  acquisition,  transfer,  retention,  strategies,  performance,  recall, 
forgetting,  testing,  and  interference  interact  in  complex  and  often  little  understood  ways,  and  more 
complex  interactions  can  be  expected  at  the  collective  or  team  level. 

Terminology  is  also  inconsistent  and  confusing.  Conceptually  it  is  often  difficult  to  determine 
whether  one  is  reading  about  tasks,  skills,  sub-tasks,  or  aptitudes.  The  tenn  “collective”  itself  is  hardly 

Human. systems®  Incorporated 


Page  84 


Collective  and  Cognitive  Skill  Retention 


ever  used,  and  no  definition  was  found,  except  in  so  wide  a  manner  (e.g.  two  or  more  people 
interacting  for  a  common  goal )  as  to  overlap  very  largely  with  definitions  of  team ,  or  organization. 
Whatever  “collective”  term  one  uses  (team,  organization,  group,  crew),  training  research  is  in  the  early 
stages  with  respect  to  either  acquisition  or  retention,  with  progress  being  made  at  different  rates. 
Research  with  respect  to  team  mental  models  appears  to  be  one  of  the  most  promising,  though  its 
translation  to  retention  issues  remains  sketchy. 

The  significance  of  predicting  the  retention  interval  for  planning  refresher  training  may  also  be 
questioned,  as  may  assumptions  about  the  level  of  performance  achieved  for  any  collective  task,  but 
especially  ones  involving  very  large  collectives.  It  seems  probable,  given  the  apparent  infrequency  of 
practice,  the  turbulence  and  turnover  in  personnel,  and  lack  of  consistency  in  the  conditions  of 
practice,  that  performance  on  collective  tasks  will  seldom  reach  the  ultimate  goal  of  training: 
autonomous  performance.  In  other  words,  collectives  or  teams  seem  likely  to  dwell,  in  normal 
training,  in  a  stage  where  procedures  are  being  learned  and  team  mental  models  built  and  the  cognitive 
or  associative  stage  may  have  been  reached  but  that  autonomous  performance  is  some  way  ahead.  In 
fact,  one  could  even  argue  that  collectives  may  not  be  able  to  reach  Anderson’s  autonomous  stage  until 
collective  membership  stabilizes,  and  until  repeated  practice  under  consistent  conditions  can  be 
undertaken. 

This  begs  the  question,  if  we  are  to  model  retention  for  the  purpose  of  assisting  the  planning  of  unit 
level  training  of  collective  tasks,  what  aspects  of  retention  are  best  modelled  and  measured.  The  most 
fruitful  approach  would  appear  to  build  on  the  team  mental  model  research  and  to  predict  retention  of 
knowledge  for  areas  outlined  in  the  current  team  mental  model  literature  among  different  team 
members,  and  retention  of  performance  for  critical  and  interdependent  tasks. 

We  would  also  argue  that  type  of  training  should  be  considered  as  part  of  the  modelling  process.  It 
seems  likely  that  training  approaches  to  maintain  individual  knowledge  and  skills  relevant  to  maintain 
team  performance  for  a  given  collective  task  at  the  cognitive  or  associative  stage  are  likely  different 
from  those  required  to  move  it  to  the  autonomous  stage. 

Talking  about  “collective  knowledge”  and  “collective  skills”  can  be  misleading.  Rather  individuals 
have  knowledge  and  skills  that  contribute  to  team  performance.  For  instance,  to  form  a  team  likely  to 
perform  well  on  a  given  collective  task,  one  would  probably  pick  individuals  with  proven  experience 
and  skill  in  the  different  individual  tasks  (especially  critical  or  interdependent  tasks),  and  in  the  likely 
context  of  application.  (To  make  the  pick  would  require  a  good  overall  knowledge  to  the  sub-tasks  and 
their  interdependencies,  and  the  context  in  which  the  collective  task  might  be  performed  -  i.e.  have 
access  to  a  comprehensive  team  task  analysis.) 

A  team  that  scores  high  on  relevant  individual  skills  and  knowledge  will  likely  perform  well  on  their 
collective  task,  even  without  practice  together.  Such  a  team  will  likely  perform  better  once  they  get  to 
know  how  each  other  react  and  have  a  chance  to  generate  a  common  idea  about  how  to  deal  with 
contextual  issues:  i.e.  plan  together. 

Moreover,  individuals  on  a  newly  formed  team  who  are  experienced  with  teamwork  and  with  the 
collective  task  in  question,  are  still  likely  to  know  what  to  look  for  in  other  team  members  and  what 
questions  to  ask  about  interdependent  and  critical  tasks  and  the  context  in  which  they  will  have  to 
work.  This  may  enable  a  team  of  experienced  individuals  to  gel  more  quickly  as  a  team.  (One  might 
describe  this  as  making  a  rapid  progression  from  one  learning  stage  to  another.)  Refresher  training  for 
such  a  group  might  look  quite  different  from  that  for  a  group  of  individuals  largely  without 
experience,  and  progress  at  a  different  rate.  If  such  a  scenario  seems  plausible,  then  the  question  is, 
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what  experience  is  relevant  for  what  collective  tasks,  and  what  type  of  training  is  most  appropriate. 
That  question  needs  careful  examination  and  empirical  data  before  any  serious  modelling  takes  place. 

The  temptation  is  strong  to  advocate  starting  completely  afresh,  but  that  approach  is  probably  what 
opened  Pandora’s  box  in  the  first  place.  However,  it  does  seem  necessary  to  step  back  and  look  at  the 
bigger  picture  to  determine  where  to  start  and  what  comes  first. 

The  first  nettles  to  grasp  seem  to  be  matters  of  scope  and  terminology.  As  a  point  of  departure,  this 
may  result,  initially,  in  a  set  of  fragmented  ideas,  but  the  groundwork  needs  to  be  laid  out  so  that  the 
elements  can  subsequently  be  brought  together. 

7.2  Scope 

In  terms  of  scope,  to  us,  the  goal  appears  to  be  essentially  a  practical  applied  one:  to  help  military 
training  planners  at  the  unit  level  determine  what  tasks  to  train,  how  and  when.  Thus,  we  should  first 
better  understand  the  context  within  which  such  planners  have  to  work  or  risk  undertaking  tangential 
research.  Priority  questions  that  come  to  mind  include  the  following.  An  important  first  step  in  any 
research  program  will  be  to  frame  the  answers  appropriately. 

•  Definition  of  the  planning  “Unit”.  Presumably  this  will  be  the  level  of  unit  at  which  there 
is  some  autonomy  over  what  to  train  and  when.  We  assume  that  this,  in  the  army,  is  at  the 
level  of  an  infantry  battalion,  or  its  equivalent,  and  in  the  navy,  a  ship. 

•  Identification  of  goals  /  needs  of  unit  planners.  Perhaps  broad  training  performance  goals 
can  be  thought  at  two  levels:  maintenance  and  mission.  In  such  a  formulation,  a  unit  is 
required  to  maintain  performance  on  a  range  of  prioritized  collective  tasks  to  some 
minimum  background  level  and  then  to  improve  performance  on  a  sub-set  of  these  tasks  to 
a  mission  standard.  This  is  probably  done  by  setting  out,  within  goals  from  higher 
authority,  an  annual  planning  schedule  that  specifies  what  tasks  each  sub-unit  will  learn  / 
practice,  how  this  is  to  be  done  (with  reference  to  training  manuals)  and  the  access  to 
resources  required. 

•  Unit  training  cycle.  Within  units,  military  training  appears  to  cycle  within  and  between 
years  for  both  individual  and  collective  tasks.  A  common  starting  point  is  the  end  of 
summer  period,  during  which  individuals  upgrade  courses,  postings  occur,  and  vacations 
are  taken.  After  this,  the  unit  re-constitutes  (to  some  degree),  first  hones  individual  skills, 
then  small  group  skills,  culminating  prior  to  the  next  summer  in  whole  unit  training,  and 
training  in  conjunction  with  other  units. 

This  within-year  cycle  may  be  supplemented  by  a  between-year  cycle  from  operational  period  to 
operational  period.  The  between-year  cycle  will  undoubtedly  vary  with  operational  demands  and  the 
time  frame  may  be  compressed  or  expanded.  A  starting  point  might  be  taken  as  end  of  a  period  on 
operations.  The  cycle  would  then  start  with  one  year  spent  re-forming  and  re-training  individual  and 
small  team  skills,  followed  by  a  year  working  up  to  the  required  performance  on  sub-unit  tasks 
culminating  in  training  on  collective  tasks  involving  other  units.  The  third  year  in  cycle  is  spent  on 
operations,  with  in-theatre  training.  As  the  second  year  progresses,  scheduling  and  planning  for 
training  would  be  guided  by  likely  priorities  for  the  coming  operations. 

If  this  outline  is  accurate,  the  challenge  for  unit  training  planners  is  to  focus  training  on  performance 
of  likely  operational  tasks,  under  mission  conditions.  Furthermore,  planning  and  prediction  issues 
such  as  the  positive  or  negative  effect  on  retention  of  other  individual  and  group  training  activities 
undertaken  during  the  retention  period  need  to  be  studied  in  the  context  of  particular  training  cycles 
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and  programs.  The  same  is  true  in  considering  the  impact  of  turbulence  and  turnover  within  and 
between  sub-units  involved  in  collective  tasks. 

•  Delimitation  of  focus  among  potential  collectives.  Leaving  definitional  problems  aside  for 
the  moment,  there  is  a  need  to  delimit  study  in  terms  of  what  sort  of  unit  we  mean  by  the 
term  collective.  Consensus  seems  unlikely,  but  aspects  might  include  the  stability  of  the 
structure,  the  number  of  personnel  in  the  collective,  and  number  of  levels  of  organization. 
For  study  purposes,  we  would  suggest  that  a  common  type  of  military  collective  should  be 
chosen,  to  ease  gathering  of  robust  data  sets  for  research  purposes. 

We  would  suggest  that,  at  least  for  the  time  being,  research  should  be  limited  to  military  groups  that 
are  relatively  small  (say  less  than  30),  have  no  more  than  two  structural  levels,  have  a  structure  that  is 
stable,  and  tend  to  endure  and  work  together  as  a  unit.  This  would  permit  research  on  collectives  to  be 
structured  around  a  more  manageable  number  of  variables,  and  would  include,  for  example,  army  units 
of  platoon  but  not  company  size,  departments  within  ships,  but  not  ships  as  a  whole10.  We  would  also 
exclude  collective  groups  that  tend  to  work  together  only  occasionally  and  formed  in  somewhat 
unpredictable  ways  from  specialized  sub-units.  This  might  include  army  combat  teams,  battle  groups, 
brigade  groups  that  comprise  different  specialist  sub-units  that  may  combined  or  re-combined  at  short 
notice,  and  seldom  train  together  in  their  entirety. 

It  may  be  possible  to  consider  these  constraints  on  collective  size  and  structure  as  a  something  of  a 
“window”  that  may  be  shifted  up  or  down  over  a  given  organizational  structure.  Something  like  this  is 
done,  for  instance,  for  staff  exercises  (so  called  CPX),  where  training  goals  are  limited  to  collective 
members  responsible  for  command  and  co-ordination  aspects  of  a  collective  task,  not  the  performance 
of  everyone  in  the  collective.  A  benefit  of  such  an  approach  would  be  to  allow  a  focus  on  C2  tasks, 
while  retaining  study  control  over  important  structural  factors  likely  to  affect  acquisition  and  retention. 
Related  sources  of  research  might  include  studies  of  management  teams  and  committees,  in  relation  to 
performance  of  collective  tasks  at  a  higher  levels  of  an  organizational  structure,  performing  more 
cognitive  tasks  of  the  problem  solving  and  decision  making  variety,  where  information  exchange  and 
co-ordination  among  critical  interdependent  sub-tasks  is  a  priority. 

7.3  Terminology 

There  are  several  terminological  issues  that  need  sorting  out,  or  at  least  fixing  clearly  for  the  purposes 
of  comparing  research  outcomes,  and  facilitating  discussion  of  the  problems  involved. 

Tasks  The  term  ‘task’  leads  to  confusion  between  the  task  of  the  collective  and  the  sub-task  of  the 
individual  within  the  collective  task.  There  needs  to  be  some  consistent  terminology  to  differentiate 
these.  This  may  be  as  simple  as  task  and  sub-task,  used  consistently.  Tasks  or  sub-tasks  can  be  seen  as 
having  some  particular  combination  of  goals,  procedures,  using  particular  equipment,  and  requiring  a 
set  of  underlying  (more  generic)  skills,  and  relating  to  the  functions  to  be  fulfilled. 

Further  distinctions  are  needed:  for  example  among  tasks  that  are  performed  largely  in  isolation  from 
other  individuals  and  those  which  depend  on  interaction  with  others. 

There  are  other  ‘  task ’  related  terms  appearing  in  the  literature  that  need  attention:  not  all  mutually 
exclusive.  Among  these  are  task  structure,  task  analysis,  task  interdependence,  task  flow,  task 


10  Departments  in  ships  may  prove  particularly  valuable  from  a  research  point  of  view,  because  the  watch  system  would 
permit  longitudinal  comparison  of  teams  under  close  to  identical  conditions  and  tasks,  taken  over  a  period  of  a  few  days. 
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criticality,  and  task  importance.  In  one  way  or  another  these  terms  are  inter-related  and  emerge  in 
different  ways  in  the  literature.  For  example,  task  importance  is  used  to  refer  to  the  potential  of  non  or 
poor  performance  to  impair  overall  job  performance,  or  affect  safety.  This  overlaps  somewhat  with  the 
idea  of  task  criticality,  where  a  sub-task  is  on  the  critical  path  for  timely  completion  of  the  overall 
collective  task.  Analysis  of  task  flow  at  the  functional  and/or  team  level  is  used  to  identify  sub-task 
interdependence,  importance,  or  criticality.  It  is  clear  that  understanding  of  certain  factors  thought 
critical  to  the  effect  of  retention  of  sub-task  performance  will  depend  on  a  preceding  team  task  analysis. 
Many  of  these  terminological  issues  are  probably  best  resolved  by  the  choice  of  a  team  task  analysis 
method. 

Skill  A  skill  may  be  understood  as  underlying  performance  on  some  task  or  sub-task,  but  being 
necessary  rather  than  sufficient  to  achieve  a  required  level  of  task  or  sub-task  performance.  Thus,  a 
very  skilled  marksman  may  not  show  appropriate  performance  on  some  sub-task  requiring 
marksmanship.  On  the  other  hand,  a  given  skill  may  underlie  performance  on  several  quite  different 
tasks.  The  models  and  literature  reviewed  sometimes  slide  disconcertingly  between  the  two  slippery 
concepts  (task  and  skill)-  but  for  the  most  part,  deal  in  “tasks”  (see  preceding  bullet).  Someone  who  is 
deemed  to  be  skilled  marksman,  for  example,  demonstrates  this  by  showing  proficiency  at  certain  sub¬ 
tasks.  Another,  sometimes  confusing,  distinction  separates  knowledge,  rules  and  skill  -  sometimes  as 
stages  of  learning,  sometimes  as  behavioural  options  contingent  on  circumstances.  Thus,  skills  can  be 
seen  as  autonomous  and  largely  unconscious  behaviour  based  on  internalized  rules  and  knowledge  and 
reached  only  after  extensive  learning  and  practice.  Where  to  place  the  dividing  line  between  a  skill  and 
a  task  or  sub-task  is  an  issue:  any  choice  will  be  somewhat  arbitrary,  but  should  be  consistent. 

Measuring  performance  on  some  task  or  sub-task  is  the  only  way  to  demonstrate  that  training  has  been 
effective  (or  retained).  Task  performance  is  the  result  required  on  the  day.  The  need  to  consider  both 
individual  and  team  performance  further  complicates  the  measurement  situation.  The  question  arises 
whether  it  is  really  meaningful  to  speak  of  collective  skills,  as  such,  as  opposed  to  measures  of  overall 
collective  performance  and  individual  proficiency  in  key  individual  tasks,  which  may  be  interactive 
(e.g.  the  task  of  co-ordination). 

Military  training  at  the  unit  and  collective  level  appears  mostly  task,  rather  than  skill  based  though, 
inevitably,  task  based  training  will  practice  generic  individual  skills.  For  collective  tasks,  refresher 
training  of  team  or  sub-task  individual  knowledge  or  skills  that  are  generic  to  several  high  priority 
collective  tasks  is  likely  to  provide  greater  return  on  training  investment  than  training  collective  task 
by  collective  task.  Interest  in  retention  then  turns  from  performance  on  collective  tasks  to 
performance  on  tests  of  common  sub-tasks,  or  retention  of  knowledge  for,  say,  procedures  related  to 
critical,  interdependent,  or  interactive  tasks.  For  this  to  be  effective,  the  relationship  among  skills  and 
tasks  needs  to  be  established  first  for  the  collective  tasks  of  interest  -  through  some  form  of  team  task 
analysis. 

Knowledge  vs.  information.  Performance  of  an  individual  or  collective  task  and  its  related  sub  tasks 
is  likely  to  require  acquisition  and  retention  of  knowledge  (tacit  or  declarative)  of  procedures  or  rules. 
The  task  (skill)  itself  may  involve  exchange,  compilation  or  modification  of  information  in  some  way, 
based  on  that  knowledge.  These  terms  carry  different  implications  that  need  to  be  clearly  separated  - 
this  not  always  the  case  in  the  literature. 

Retention  can  be  addressed  at  a  number  of  different  levels  (possible  related  to  qualitatively  different 
plateaux  of  learning  or  decay).  One  level  is  the  retention  of  the  knowledge  needed  by  the  team  to 
perform  their  collective  task  (i.e.  mental  models  among  the  team  for  task  procedures,  capabilities  and 
roles  of  others,  etc).  This  knowledge  may,  in  turn,  be  related  to  the  retention  of  autonomous 
performance  on  the  sub-task  or  task.  If  so,  then  study  of  retention  for  both  knowledge  and 
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performance,  may  permit  more  surgical  diagnosis  of  retention  intervals  for  the  different  components 
and  related  refresher  training  needs.  For  example,  classroom  refresher  training  of  declarative  and/or 
tacit  knowledge  related  to  team  mental  model  components  for  a  given  collective  task  for  a  given  class 
of  collective  member  may  represent  a  more  cost  effective  approach  to  maintaining  overall  collective 
performance  over  longer  intervals  in  the  field.  To  answer  this  requires  empirical  data  on  such 
relationships.  Retention  curves  related  to  turnover  and  turbulence  may  also  be  susceptible  to  this  sort 
of  examination:  i.e.  periodic  updating  of  the  team  mental  models  of  current  team  members  with  respect 
to  the  roles  and  capabilities  of  new  and  old  team  members,  for  the  collective  task  of  interest. 

Measures  and  indices.  Appropriate  and  accepted  measures  and  indices  of  retention  of  different  aspects 
of  collective  performance  remain  to  be  determined.  Issues  include  whether  to  pitch  measures  at  the  level 
of  group  or  individual  performance,  how  to  combine  or  separate  speed  and  accuracy  measures,  whether 
to  focus  on  both  knowledge  and  performance,  and  process  or  outcome  measures. 

Another  question  is  how  any  measures  might  relate  to  qualitative  differences  in  the  stages  of  learning 
(and  decay?)  and,  in  turn,  to  prescriptions  for  refresher  training.  In  other  words,  is  retention  to  be 
measured  in  terms  of  a  percentage  of  earlier  performance,  or  in  terms  of  the  level  of  regression  from  a 
more  or  less  unconscious  autonomous  stage  back  to  an  associative  stage  dependent  on  declarative 
knowledge.  There  is  also  the  need  to  know  what  level  of  learning  is  to  serve  as  the  starting  point  for 
retention.  As  noted  earlier,  collectives  have  been  described  as  in  a  constant  state  of  change  in  terms  of 
membership,  and  collective  tasks  as  being  practised  over  long  intervals  under  very  different 
conditions.  This  raises  the  question  of  whether  collective  tasks,  even  simple  ones  for  small 
collectives,  ever  reach  the  autonomous  stage  of  learning,  except  after  a  period  of  continuous 
operations  with  constant  team  membership  for  a  limited  range  of  collective  tasks. 

With  limited  training  time  available,  and  limited  opportunities  for  training  as  a  collective  as  whole 
under  realistic  field  conditions,  what  will  provide  the  greatest  (or  any)  return  on  refresher  training? 
Would  it  be  best  to  refresh  selected  individual  tasks  on  the  critical  path  for  collective  performance  or 
on  interactive  tasks  performed  among  selected  members  of  the  collective.  Will  knowledge  comprising 
the  different  elements  of  team  mental  models  decay  at  the  same  rates,  and  will  refreshing  of  different 
knowledge  sets  contribute  to  retention  of  performance  on  the  overall  collective  task?  Answers  to  all 
these  questions  are  needed  as  the  basis  for  an  effective  model  and  to  support  planning  of  collective 
refresher  training. 

Cognitive  skills  /  tasks.  The  term  “cognitive”  is  used  in  different  ways  in  the  literature.  On  the  one 
hand,  it  is  used  to  refer  to  a  class  of  tasks  such  as  problem  solving  and  decision-making  that  emphasize 
manipulation  of  information  in  some  form.  This  description  appears  to  be  reserved  for  more  complex 
tasks  that  are  difficult  to  proceduralize,  and  often  contrasted  with  simple  perceptual  motor  tasks.  On 
the  other  hand,  the  term  is  also  used  to  refer  to  a  set  of  presumed  mental  resources  employed  in  any 
task,  such  as  attention,  memory,  and  information  handling.  The  way  in  which  cognitive  aspects  of 
collective  tasks  are  to  be  defined  needs  to  be  established  for  conceptualisation  of  future  collective 
training  research,  and  to  know  how  to  interpret  existing  research.  Our  viewpoint  is  that  cognition  is  an 
aspect  of  many  collective  sub-tasks  and,  likely,  all  collective  tasks  (if  only  for  co-ordination  purposes), 
and  not  just  selected  collective  tasks.  Thus  cognitive  issues  are  best  dealt  with  as  an  integral  part  of 
modelling  collective  tasks,  not  as  a  separate  category. 

Communication  and  information  exchange.  In  military  terms,  communication  has  often  been 
equated  with  radio-based  skills  such  as  voice  procedures,  and  network  knowledge.  In  functional 
terms,  radio  communication  is  simply  one  method  of  exchanging  or  acquiring  information,  with 
certain  advantages  and  disadvantages.  Effective  information  exchange  involves  much  more  than 
working  a  radio  net  and  lies  increasingly  at  the  heart  of  many  collective  tasks  as  sensor,  software,  and 
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communication  technologies  advance.  The  concept  of  information  exchange  should  probably  be 
broadened  to  incorporate  all  aspects  of  information  handling  within  a  collective:  for  both  individual 
and  interactive  sub-tasks  that  affect  overall  collective  performance. 

Team,  group,  organization,  collective,  unit.  The  concepts  and  definitions  associated  with  these  terms 
overlap  to  such  a  degree  that  one  forced  to  wonder  about  the  need  for  the  term  “collective”  at  all.  The 
effect  of  its  use  in  this  project  was  to  confound  the  search  for  relevant  material:  most  of  what  was 
found  related  to  teams,  small  groups,  or  organizations.  No  distinguishing  feature  of  “collectives”  was 
found  to  justify  another  term.  The  only  unclaimed  conceptual  territory  appears  to  be  “a  loose  and 
temporary  grouping  of  different  organizations  or  teams  of  differing  sizes  or  specialities  to  achieve  a 
common  goal”:  But  this  is  not  the  sense  the  term  has  been  used  in  the  few  “collective”  articles  found 
in  this  search  or,  apparently,  in  their  secondary  references. 

We  believe  that  interest  and  conceptualisation  should  be  refocused  on  the  more  widely  accepted  terms  of 
“organization”  and  “team”  and  the  term  “collective”  either  more  systematically  related  to  these  other 
concepts  or,  more  likely,  discarded  altogether.  This  would  allow  the  net  to  be  cast  wider  for  existing 
research  on  issues  relating  to,  for  instance,  organizational  rather  than  collective  structure  and  the  wide 
literature  on  effectiveness  of  management  and  other  teams,  the  surface  of  which  was  only  scratched  in 
this  literature  review.  It  may  also  reveal  models  of  organizational  and  team  structure  that  can  be  put  to 
good  use. 

7.4  Existing  approaches 

In  general,  none  of  the  existing  models  can  be  considered  adequate  to  use,  on  their  own,  for  predicting 
retention  of  collective  performance. 

•  The  Nichols  model  is  simply  not  complex  enough  to  be  able  to  account  for  all  the 
important  factors. 

•  The  UDA  model  of  individual  skill  retention  is  much  more  elaborate,  and  the  factors  in  that 
model  might  be  adapted  to  the  issue  of  collective  performance,  especially  where  individual 
tasks  critical  to  collective  performance  are  concerned.  More  recent  efforts  to  extend  the 
UDA  model  encourage  that  conclusion,  but  do  not  go  far  enough  to  capture  the  broad  range 
of  factors  influencing  retention  of  collective  performance.  More  critically,  as  far  as  we  could 
tell,  the  UDA  model  still  needs  broader  validation. 

•  The  Roth  model  (1992)  is  the  most  comprehensive  model  dealing  with  collective  performance 
that  we  found.  However,  despite  its  conceptual  promise,  the  Roth  model  omits  potentially 
important  factors.  Also,  validation  is  very  limited  and  subjectively  based,  and,  as  far  as  it 
goes,  does  not  support  the  model  as  a  good  predictor  of  collective  skill  retention. 


7.5  A  research  program 

Looking  at  a  way  ahead,  we  have  chosen  to  present  this  in  the  form  of  an  evolutionary  3  year  research 
program  based  on  the  assumed  structure  of  a  one  year  military  training  cycle.  The  overall  purpose 
would  be  to  gather  empirical  data  on  retention  of  team  performance,  develop  methods  of  analysing 
team  performance  issues,  and  to  model  retention  of  team  performance  and  relate  this  to  the  provision 
of  planning  support  for  unit  level  training. 

To  achieve  these  goals,  we  advocate  a  narrowly  focussed  longitudinal  approach  set  in  the  overall 
context  of  successive  training  years  and  believe  that  model  building  should  be  interleaved  with 
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gathering  of  empirical  data  on  two,  small  and  representative  collective  structures,  and  two  contrasting 
but  common  collective  tasks. 

For  the  purposes  of  the  current  work,  data  should  be  gathered  for  two  actual  collectives  on  the  range  of 
practical  predictive  factors  outlined  in  this  report.  This  should  be  done  in  the  context  of  the  annual  (or 
longer)  unit  training  program  for  the  specific  collectives  chosen  in  order  to  examine  the  impact  of  as  full 
a  range  of  factors  as  possible  operating  during  any  given  retention  interval.  Data  should  be  gathered  on 
retention  of  the  overall  collective  task  and  sub-tasks  in  terms  of  speed  and  accuracy  of  performance,  and 
retention  of  team  related  knowledge  for  the  collective  tasks  chosen. 

The  foundations  of  the  approach  proposed  would  be  a  detailed  analysis  of  the  training  cycle,  the 
collective  structures,  and  the  collective  tasks.  A  detailed  team  task  analysis  would  identify  critical 
individual,  interdependent,  and  interactive  sub-tasks.  Retention  of  performance  on  these  would  then  be 
examined  in  relation  to  a  given  annual  training  program,  and  an  estimate  obtained  of  different  elements  of 
the  program  (such  as  training  on  individual  skills  and  related  collective  tasks)  the  likely  impact  on 
different  aspects  of  retention  of  individual  knowledge  and  collective  performance.  Retention  of  actual 
performance  and  knowledge  would  be  tracked  at  appropriate  intervals  as  well  as  other  potential  influences 
on  the  retention  of  these  tasks  over  the  training  year,  such  as  turnover  and  turbulence. 

The  specific  goals  of  this  work  would  be,  for  the  collectives  and  collective  tasks  selected,  to: 

•  Analyse  needs  of  unit  training  planners. 

•  Analyse  annual  training  cycles  with  respect  to  training  collective  tasks. 

•  Identify  and  apply  a  team  task  analysis  approach  to  determine  sub-task  criticality  and 
interdependence,  and  the  nature  of  cognitive  and  interactive  skills  required. 

•  Identify  the  knowledge  required  for  an  effective  team  mental  model. 

•  Gather  data  on  changes  in  collective  task  performance  over  the  annual  training  cycle  and 
relate  to  selected  factors  thought  likely  to  influence  retention. 

•  Examine  the  applicability  of  existing  models  of  individual  and  team  retention  to  critical 
collective  sub-tasks. 

•  Study  the  effect  of  factors  related  to  the  retention  interval  on  retention  of  performance  on 
individual  knowledge,  and  critical  and  interactive  sub-tasks. 

•  Study  the  effect  of  other  factors:  include  task  and  sub-task  factors,  collective  structure, 
individual  differences  among  collective  members,  and  training  methods. 

•  Examine  the  practicalities  of  gathering  and  using  existing  data  to  study  predictive  factors 
such  as  membership  turbulence  and  turnover  and  individual  differences.. 

•  Frame,  evolve  and  empirically  validate  a  model  for  predicting  retention  of  performance  of 
collective  tasks  and  predicting  related  training  requirements. 

Use  task  modelling  software  such  as  “Micro- Saint”  as  a  tool  to  examine  the  potential 
impact  of  different  retention  on  critical,  interactive,  and  interdependent  sub-tasks 

•  Develop  and  evaluate  a  decision  support  tool  for  unit  planners  for  collective  training. 


These  goals  and  approaches  are  outlined  as  a  three  year  plan,  in  the  table  below. 


Item 

Comments 

YEAR 

Analyse  unit  training  cycle  and  planning. 

May  exist  in  literature 

Humanly  .stem  .s’  Incorporated 
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ONE 

Extend  literature  search/review  of  team 

Choose  a  suitable  team  task 

Planning 

&  organizational  structure  issues. 

analysis  method. 

Establish  recommendations  for 

and 

team  training  approaches. 

preparation 

Choose  two  contrasting  team  structures 

Relate  to  retention  factors 

Choose  two  contrasting  team  tasks. 

Relate  to  retention  factors 

Conduct  team  task  analysis  to  determine: 

-critical  individual  tasks 

-  interdependent  tasks 

-  Interactive  tasks 

-  measures  of  team  and  individual  task  performance  (MOPS) 

-  knowledge  required  for  team  mental  model  for  task. 

Determine  access  to  turnover  and  turbulence  data 

Obtain  /  analyse  unit  annual  training  plan  to  predict  effects  on  target 
team  and  individual  tasks. 

September  start  date? 

Identify  specific  units  to  be  studied. 

Use  UDA  /  Nichols  /  Roth  models  to  predict  retention  of  key  sub-tasks. 

Establish  recommendations  for  team  training  approaches. 

Based  on  extended  literature 

YEAR 

Identify  background  aptitudes  of  team  members. 

Use  recruit  aptitude  scores? 

TWO 

Measure  individual  and  team  task  performance  at  start. 

Initial 

data 

Test  performance  at  standard  intervals 

Once  a  month? 

Track  turbulence  and  turnover  over  training  year 

gathering. 

Track  all  training  received  and  relate  to  team  tasks  and  sub-tasks. 

Analyse  data  in  terms: 

-  impact  of  factors 

-  predictions  from  UDA  /  Roth  /  Nichols 

Refine  team  retention  model. 

Draft  annual  planning  tool. 

Build  on  tools  in  literature 

YEAR 

Plan  idealized  training  cycle  for  two  tasks 

THREE 

Conduct  training  using  that  plan 

Controlled  comparison 

Controlled 

data 

Track  performance  as  before  for  target  team  and  control  team. 

Assess  UDA,  Roth,  Nichols 

Refine  model. 

gathering. 

Refine  planning  tool 

Table  20:  A  three  year  program  of  research 


7.5.1  Identify  a  research  context 

The  research  context  needs  to  set  in  terms  of  the  collective  structure,  collective  task,  and  training 
program. 

Collective  structure.  For  practical  and  economic  as  well  as  study  design  reasons,  we  advocate 
conducting  initial  work  in  some  depth  but  on  a  limited  number  of  collective  types.  The  program 
should  start  with  as  few  as  two  types  of  collective  that  represent  a  contrast  in  terms  of  influential 
structural  factors  such  as  the  number  sub-teams,  the  number  of  organizational  levels,  and  /or  the 
degree  of  specialisation  required.  Ease  of  access  for  repeated  longitudinal  data  gathering  should  also 
feature  in  the  choice,  with  common  /  numerous  collectives  having  a  higher  priority  for  choice.  In 
Canada,  this  might  result  in  collectives  such  as  infantry  platoons,  armour  or  artillery  troops  being  the 
most  probable  candidates. 
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However,  the  choice  should  consider  issues  related  to  gathering  of  viable  empirical  data.  Although 
army  units  might  be  the  current  focus  of  priority,  looking  from  a  tri-service  point  of  view  merits 
consideration.  Theoretical  issues  should  remain  constant  across  all  applications.  Instances  why 
inclusion  of  study  of  small  navy  teams  might  have  advantages  include  the  naval  watch  system  which 
permits  study  of  different  teams  under  identical  conditions  for  identical  tasks,  and  the  existence  of 
sophisticated  navy  team  training  simulators  used  for  both  initial  work-ups  among  ship  teams  (i.e.  to 
reach  initial  criterion)  and  re-fresher  training  prior  to  departure  on  a  mission.  Incorporating  team 
training  measures  as  standard  practice  for  all  simulator  training  and  archiving  the  results  might  greatly 
benefit  long  term  research. 

Initially,  statistical  issues  should  probably  be  treated  in  qualitative  and  descriptive  terms,  but  later 
research  and  validation  will  need  to  consider  analytical  statistics  and  sample  sizes  -  perhaps 
comparing  across  contributing  countries.  Choice  of  a  commonly  occurring  type  of  collective  group 
should  ease  later  statistical  challenges.  However,  the  choice  should  be  made  after  a  systematic  review 
of  the  options.  Later,  if  some  fundamental  relationships  can  be  established,  the  range  of  collective 
type  may  be  expanded. 

Collective  tasks.  For  the  same  reasons  as  above,  the  program  should  start  with  two  contrasting  types 
of  collective  task,  representing,  for  instance,  different  degrees  of  interdependence  among  sub-tasks. 

The  range  of  tasks  under  consideration  will  depend  on  the  type  of  collectives  selected.  Once  again  a 
systematic  review  of  the  options  should  be  conducted.  Tasks  for  which  ready  access  to  suitable 
training  simulators  exists  for  evaluating  individual  tasks  offer  potential  advantages. 

Training  program  There  are  three  reasons  for  needing  to  set  the  study  in  the  context  of  the  overall 
training  cycle  over  which  collective  tasks  have  to  be  retained.  First,  the  global  goal  is  to  provide  decision 
support  for  planners  of  unit  training  programs.  This  significance  of  this  point  has  been  discussed 
elsewhere  in  this  section.  Second,  many  potential  factors  relate  to  events  during  the  retention  interval,  in 
particular  for  positive  and  negative  transfer  of  training  from  other  training  on  individual  and  collective 
tasks  but  also  in  terms  of  changes  in  the  membership  of  the  collective.  The  natural  choice  appears  to  be  to 
use  the  context  of  a  specific  annual  training  cycle  for  a  specific  major  unit,  within  which  the  training  of  the 
collective  type  and  task  of  choice  is  conducted.  The  third  reason  is  to  take  advantage  of  the  already 
extensive  training  documentation  available  in  most  military  settings  with  respect  to  description  of  training 
objectives,  standards  of  evaluation,  training  manuals,  simulation  facilities,  and,  possibly,  pre  -existing  data 
bases  on  matters  relating  to  the  research  questions  posed,  particularly  profiles  of  collective  members.  The 
latter  might  include  aptitude  data,  records  of  qualifications  and  training  results,  operational  experience, 
computerized  records  from  which  turbulence  and  turnover  data  can  be  derived,  and  others. 

7.5.2  Extend  knowledge  of  existing  research 

Concerns  have  been  expressed  several  times  in  the  report  that  only  the  surface  has  been  scratched  for 
relevant  organizational  design,  communication  behaviour,  and  team  work  research.  We  strongly 
recommend  extending  the  literature  search  to  focus  on  such  issues  as  part  of  the  preparation  for  this 
research  program.  In  particular,  we  recommend  a  closer  examination  of  research  into  team  training  and 
team  mental  models. 

Review  of  existing  approaches  to  optimising  training  strategies  will  also  merit  investment  with  respect 
to  providing  support  to  unit  training  planners.  Work  by  Matto  and  Moses  (1997),  for  example, 
presents  a  procedure  called  Training  Strategies  Optimisation  Prototype  (TSOP)  to  be  used  by  unit 
level  commanders  in  order  to  optimisation  proficiency  at  a  collective  level.  Employing  this  strategy 
involves  feeding  in  specific  inputs,  including  initial  proficiency,  training  method  utility,  skill 


Wummsystems®  Incorporated 


Collective  and  Cognitive  Skill  Retention 


Page  93 


degradation,  training  method  costs  and  resource  availability.  The  output  of  the  approach  is 
information  about  the  optimal  training  strategy  given  the  inputs.  Estimates  of  initial  proficiency, 
however,  are  based  on  a  commander’s  view  of  skills  as  having  been  trained,  being  in  need  of  practice, 
or  being  untrained.  This  fairly  broad  and  subjective  system  of  assessment  may  well  lack  the  degree  of 
precision  needed  but  consideration  of  such  approaches  are  potentially  important  in  the  context  of  the 
applied  goal  of  this  research. 

7.5.3  Analyse  collective  team  and  task  structure 

It  is  clear  that  any  future  research  needs  to  be  set  against  a  better  understanding  of  the  nature  of  the  sub¬ 
tasks  (especially  in  relation  to  their  cognitive  aspects)  in  any  overall  collective  task.  Sub-tasks  need  to  be 
identified  and  considered  in  terms  of  their  acquisition  and  retention,  relevant  knowledge, 
interdependence,  and  criticality.  The  presence  of  interaction  among  individuals  and  within  and  between 
sub-groups  will  also  be  important,  as  will  be  the  level  of  information  exchange.  Without  a 
comprehensive  understanding  of  the  way  in  which  a  collective  task  knits  together  and  the  way  in  which 
its  completion  relates  to  its  sub-tasks,  any  future  research  is  likely  to  be  fruitless.  This  implies  the  need 
for  a  comprehensive  team  task  analysis.  An  early  step  in  any  research  therefore  would  be  to  establish  a 
suitable  team  task  analysis  method,  and  apply  it. 

The  use  of  flowchart  and  swim  lane  depiction  methods  to  analyse  team  or  collective  tasks  is  not  new. 
Such  approaches  have  long  been  advocated  as  a  method  to  depict  the  processes  that  teams  use  during 
the  completion  of  missions  and  for  project  planning  using  approaches  such  as  critical  path  analysis. 

For  example,  Microsoft  Project  appears  to  have  the  potential  to  form  the  basis  for  some  aspects  of  the 
required  analysis. 

A  conventional  task  analysis  or  project  management  approach,  however,  will  only  go  so  far  in 
understanding  a  collective  task.  What  is  also  required  is  a  way  to  represent  the  sequence  of  steps  that, 
taken  together,  comprise  a  collective  task.  What  is  needed  is  a  method  to  capture  individual  sub-tasks 
as  well  as  the  task  interdependencies  related  to  roles  within  the  collective  (Salas  and  Cannon-Bowers, 
2000;  in  Paris  et  al.,  2000).  It  will  also  be  important  to  understand  cognitive  components  of  collective 
sub-tasks  using  some  form  of  cognitive  task  analysis  at  the  team  level. 

For  this  reason,  an  important  step  in  the  research  program  will  be  to  identify  suitable  task  analysis 
methods.  A  frequent  criticism  of  applying  conventional  task  analysis  methods  to  a  team  is  that  critical 
teamwork  behaviours  (e.g.  interdependence,  co  -ordination  etc.)  are  not  included  (Brenner  et  al., 

1998).  Other  authors  believe  that  task  analysis  may  be  appropriate  within  teams  if  it  establishes  a  link 
across  job  tasks  (Dieterly,  1998). 

Paris  et  al.  (2000)  argue  that  capturing  skills  within  teams  requires  an  approach  that  is  multilevel  and 
dynamic.  Team  task  analysis  is  the  first  necessary  step.  This  process  provides  information  about  team 
learning  objectives  and  competencies,  and  specifies  the  cues,  events,  actions,  co-ordination  needs  and 
communication  flows  for  good  teamwork.  Moreover,  a  team  task  analysis  promotes  an  understanding 
of  interdependency  and  works  to  distinguish  collective  team  tasks  from  the  individual  tasks.  Several 
approaches  have  been  enacted  in  efforts  to  better  measure  team  skills  (e.g.  Annett,  Cunningham  and 
Mathias-Jones,  2000).  These  need  to  be  examined  and  a  choice  made. 

The  measurement  of  communication  and  co-ordination  only  addresses  the  issue  of  observable  behaviour, 
and  leaves  the  larger  area  of  the  inferred  processes  (e.g.  shared  mental  models)  untouched.  The  extent  to 
which  this  matters  may  depend  on  whether  the  focus  is  on  measuring  learning  (and  retention  of  learning) 
or  on  performance.  There  may  be  many  military  contexts  (e.g.  with  tank  crews)  where  the  performance 
of  a  given  task  is  all  that  matters.  Either  a  gunnery  crew  is  able  to  hit  the  target  or  not.  In  a  more 
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complex  context,  non-observable  processes  such  as  decision  making  and  the  degree  to  which  collective 
members  share  mental  models  may  be  a  better  predictor  of  retention  of  collective  performance. 

Annett,  Cunningham  and  Mathias-Jones  (2000)  employ  Hierarchical  Task  Analysis  for  Teams  to 
identify  the  components  of  tasks  that  must  be  completed  or  are  best  completed  via  teamwork. 
Teamwork,  in  this  context,  is  loosely  defined  as  “ communication  between,  and  the  co-ordination  of 
actions  of,  members  of  the  team”. 

7.5.4  Create  suitable  measures  and  indices 

Consideration  is  needed  of  suitable  measures  and  indices  for: 

•  Factors  to  be  considered  in  modelling 

(features  of  collective  tasks,  collective  structure,  individual  differences,  training) 

•  Performance  on  collective  tasks  and  individual  sub-tasks  for  speed  and  accuracy 

These  must  provide  both  a  practical  means  by  which  to  gather  data  for  research  and  for  unit  training 
planners,  as  well  as  provide  utility  for  research  and  for  a  decision  support  tool  for  unit  planners. 

Preliminary  planning  should  select  or  create  suitable  evaluation  criteria,  measures,  methods  and 
standards.  Some  may  be  drawn  from  the  military  training  literature  for  the  collective  tasks  chosen. 
Others  will  have  to  be  developed  for  the  purpose  of  assisting  unit  planners  determine  training 
strategies  and  schedules,  or  to  examine  underlying  research  issues. 

A  particular  challenge  will  be  obtaining  measures  of  team  performance.  Team  performance  has  been 
measured  along  two  main  dimensions  (Paris,  Salas,  Cannon-Bowers,  2000),  using  either  measures  of 
an  individual's  ability  to  perform  specific  tasks,  or  with  team  measures,  which  focus  on  co-ordination 
requirements  and  between  team  measures  for  given  tasks  (e.g.  backup,  information  flow,  correction  of 
errors).  Teamwork  measures  also  frequently  make  the  distinction  between  outcome  measures  of 
performance  (were  task  objectives  successful?)  and  process  measures  (how  was  the  task  completed?). 
For  C2  work,  process  measures  have  been  shown  to  be  more  practical  to  control,  measure  and  interpret 
(Matthews  and  Webb,  1997)  with  information  exchange,  situation  awareness  and  decision  making 
identified  as  key  processes. 

There  are  a  number  of  potential  sources  of  error  in  the  measurement  of  collective  performance  which 
can  be  considered  (Tumage  et  al.,  1990).  A  particular  problem  for  retention  at  the  collective  level  is  the 
difficulty  of  assessing  the  stability  of  skill  acquisition  curves.  Instability  of  the  curves  means  suggests 
variance  in  measures  unrelated  to  skill  loss  and  due  to  varying  levels  of  acquisition.  Within  teams, 
Tumage  et  al.  (1990)  argue  that  any  measure  of  team  performance  is  actually  comprised  of  3  separate 
individual  components.  Each  of  these  factors  contributes  both  true  and  error  variance  toward  the 
measure  of  collective  performance. 

•  proficiency  of  individual  team  members  on  individual  tasks 

•  proficiency  of  individual  team  members  on  team  tasks 

•  learning  as  a  result  of  the  continuing  practice  of  the  team  as  a  unit 

Team  output  measures  may  also  be  unduly  affected  by  the  performance  of  the  “weakest  link”  in  the 
team  in  terms  of  performance  on  critical  sub-tasks  that  may  reflect  problems  unrelated  to  team 
performance  such  as  equipment  problems.  Turnage  et  al.  (1990)  specifically  addresses  the  complexities 
of  performance  measurement  with  respect  to  military  training,  and  this  work  provides  a  starting  point 
from  which  need  to  be  considered  when  consider  collective  performance  measurement.  The  ability  to 
understand  collective  retention  will  depend,  in  part,  on  an  adequate  systems  of  measurement  for 
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collective  performance  in  general.  This  system  could  apply  to  a  wide  variety  of  military  specialities. 
According  to  Tumage  et  al.  ( 1 990)  such  a  system  does  not  exist. 

7.6  Conclusion 

A  search  of  the  relevant  technical  databases  for  studies  of  collective  skill  retention  revealed  only  one 
directly  relevant  research  report.  This  and  other  models  of  individual  skill  retention  were  examined 
against  the  background  of  military  training  and  the  long  standing,  wide  and  often  contradictory 
literature  on  acquisition  and  retention  of  individual  performance. 

The  broad  conclusion  of  this  work  is  that  collective  performance  should  be  considered  in  terms  of 
individual  sub  tasks  as  well  as  overall  collective  performance.  Individual  sub-tasks  include  those 
performed  in  isolation  by  collective  members  and  sub-tasks  requiring  interaction  among  collective 
members.  The  effect  of  these  individual  tasks  on  overall  collective  performance  will  depend  on  the 
criticality  and  level  of  interdependence  of  these  sub-tasks  with  respect  to  the  overall  collective  task. 
Understanding  these  relationships  among  particular  collective  tasks  for  particular  collective  structures 
is  a  pre-requisite  of  effective  study  of  retention  of  collective  performance,  and  requires  selection  or 
development  of  a  suitable  team  task  analysis  approach. 

There  are  large  of  number  of  factors  with  the  potential  to  affect  both  acquisition  and  retention  of 
performance  on  collective  tasks  that  may  be  categorized  as  those  affecting  the  collective  task  and  its 
sub-tasks,  collective  structure,  the  collective  membership,  individual  differences,  and  the  type  of 
training  provided  before  and  during  the  retention  interval  on  the  task  of  interest  and  others. 

Examination  of  the  existing  modelling  approaches  concluded  that  these  are  neither  comprehensive 
enough  in  scope  nor  sufficiently  well  validated  to  use  as  a  solid  foundation  for  future  work  on 
predicting  retention  of  collective  performance. 

A  limited  review  of  the  broader  literature  on  team  and  organizational  issues  related  to  the  acquisition  and 
retention  of  team  skills,  concludes  that  use  of  the  term  “collective”  is  very  rare,  and  appears  to  contribute 
little  added  value  to  examination  of  team  and  organizational  issues  and,  rather,  distracts  attention  from 
the  already  extensive  literature  in  these  areas.  In  particular  research  on  team  mental  models  and  their 
relation  ship  to  training  of  team  performance  is  an  area  that  merits  examination  in  more  detail  as  the 
foundation  of  research  into  retention  of  team  performance. 

To  fulfil  the  global  goal  of  developing  an  effective  decision  support  tool  for  unit  planners  and 
modelling  retention  of  team  performance  under  conditions  typical  of  military  training,  a  three  year 
research  effort  is  recommended  with  the  following  features: 

•  Extend  the  literature  review  into  related  team  and  organizational  issues. 

•  Select  or  develop  a  suitable  method  for  team  task  analysis. 

•  Analyse  military  unit  training  cycles  and  the  needs  of  unit  training  planners  in  detail, 

•  Establish  baseline  data  on  internal  turbulence  and  external  turnover  in  military  units. 

•  Develop  appropriate  measures  of  performance  for  individual  and  team  retention. 

•  Conduct  a  longitudinal  study  over  a  unit  training  cycle  of  two  contrasting  collective  types 
for  two  contrasting  collective  tasks. 

•  Use  data  gathered  to  build  a  model  of  retention  of  team  performance  and  to  evaluate  the 
potential  contribution  of  existing  models  of  individual  retention  to  predict  retention  of 
collective  performance. 
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•  Validate  this  model  in  the  second  year  of  study 

•  Develop  and  evaluate  a  decision  support  tool  for  unit  training  planners 
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14.  ABSTRACT 


(U)  This  review  was  conducted  under  the  direction  of  DCIEM  (Defence  and  Civil  Institute  of 
Environmental  Medicine).  Within  the  ultimate  goal  of  fielding  a  validated  model  to  be  used  by  military 
personnel  to  schedule  training  of  collective  tasks,  the  purpose  of  this  project  was  to  review  publications 
from  the  military  research  and  development  community  concerning  retention  of  collective  and  cognitive 
skills.  Contributions  from  the  general  scientific  literature  were  also  considered. 

The  report  is  organised  around  sections  considering  training  in  the  military;  the  nature  of  collectives  and 
collective  tasks;  development  and  retention  of  collective  perfonnance;  factors  affecting  retention  of 
collective  perfonnance;  existing  models  of  collective  performance;  and  recommendations  for  a  way 
ahead. 

A  broad  range  of  factors  with  the  potential  to  affect  retention  of  perfonnance  on  collective  tasks  was 
considered.  These  factors  were  categorised  as  those  affecting  the  collective  task  and  its  sub-tasks, 
collective  structure,  the  collective  membership,  individual  differences,  and  the  type  of  training  provided 
before  and  during  the  retention  interval  on  the  task  of  interest  and  others. 

A  brief  review  and  discussion  of  the  general  literature  on  team  work,  team  mental  models  and 
organisational  issues  suggests  that  retention  of  performance  on  individual  sub-tasks  will  be  strongly 
related  to  retention  of  overall  collective  performance  -  particularly  with  respect  to  co-ordination. 
Individual  sub-tasks  include  both  those  performed  in  isolation  by  collective  members  and  sub-tasks 
requiring  interaction  among  collective  members.  The  effect  of  these  individual  sub-tasks  on  overall 
collective  perfonnance  will  depend  on  their  criticality  and  level  of  inter-dependence  with  respect  to  the 
overall  collective  task,  and  the  need  for  information  exchange  and  interaction  with  other  collective 
members.  Understanding  the  relationships  among  sub-tasks  within  particular  collective  tasks  for 
particular  collective  structures  is  a  pre-requisite  of  effective  study  of  retention  of  collective  performance, 
and  requires  selection  or  development  of  a  suitable  team  task  analysis  approach. 

Only  one  developed  model  of  collective  skill  retention  was  found  as  a  result  of  the  literature  search.  This 
model  was  reviewed  in  conjunction  with  other  models  related  to  retention  of  individual  skills.  The 
conclusion  was  drawn  that  while  these  models  fonn  a  good  conceptual  point  of  departure,  none  are  broad 
enough  in  scope  in  terms  of  the  factors  covered  or  sufficiently  well  validated  to  serve  as  a  satisfactory 
foundation  for  predicting  retention  of  collective  performance.  Particular  areas  of  omission  included 
transfer  from  other  training  during  the  retention  interval,  experience  of  team  members,  internal 
turbulence  and  external  turnover,  and  shared  team  mental  models. 

The  term  “collective”  was  rarely  found  in  the  literature  on  training  and  its  use  appears  to  add  little  to 
terms  such  as  team  and  organisation,  already  in  common  use.  It  is  recommended  that  the  literature  on 
team  and  organisational  issues  be  more  thoroughly  reviewed  with  respect  to  the  issues  contained  in  this 
report.  In  particular,  the  literature  on  team  mental  models  should  be  considered  in  greater  detail  in 
relation  to  retention  issues,  including  retention  of  knowledge  contributing  to  team  performance. 

To  fulfil  the  global  goal  of  developing  an  effective  decision  support  tool  for  unit  planners  and  modelling 
retention  of  team  perfonnance  under  conditions  typical  of  military  training,  a  longitudinal  study  of  a 
small  number  of  contrasting  collectives  and  collective  tasks  is  recommended.  This  should  be  based  on 
analysis  of  both  the  needs  of  unit  training  planners  and  the  team  tasks  involved  in  the  collective  tasks 
chosen. 

(U)  II  est  crucial  d’offrir  le  bon  degre  de  formation,  et  il  faut  done  une  base  scientifique  pour 
Tetablissement  des  calendriers  des  cours  de  recyclage  ou  de  maintien.  II  existe  certains  ecrits 
scientifiques  sur  Tetablissement  de  calendriers  pour  la  formation  individuelle  portant  sur  des  taches 
procedurales,  mais  les  FC  fonctionnent  au  niveau  organisationnel,  en  faisant  appel  a  une  technologie 
toujours  plus  perfectionnee  et  en  accordant  de  plus  en  plus  d’ importance  aux  taches  cognitives.  Get 
article  passe  en  revue  les  donnees  et  theories  publiees  et  non  publiees  provenant  du  milieu  de  la 
recherche  et  du  developpement  militaires  et  portant  sur  la  preservation  des  competences  collectives  et 
cognitives  en  tant  qu’etape  menant  a  la  proposition  d’un  modele  valide  de  maintien  des  competences 
collectives  et  cognitives.  On  a  consulte  des  sources  electroniques  et  un  groupe  international  de 
scientifiques  en  vue  d’identifier  les  donnees  disponibles  pour  la  recension.  L’analyse  de  la 
documentation  selectionnee  a  revele  Texistence  de  cinq  facteurs  :  les  variables  des  taches,  la  structure  du 
collectif,  les  membres  du  collectif,  les  differences  individuelles  et  la  formation.  On  n’a  releve  qu’un 
modele  detaille  de  maintien  de  competences  collectives.  Ce  modele  a  ete  analyse  de  pair  avec  d’autres 
modeles  associes  au  maintien  de  comnetences  individuelles.  Aucun  n’a  une  nortee  suffisamment  vaste  en 


termes  de  facteurs  pris  en  compte  ou  n’est  suffisamment  valide  pour  servir  de  fondement  a  la  prevision 
du  maintien  du  rendement  collectif.  Un  plan  d’ elaboration  d’un  modele  de  maintien  des  competences 
collectives  est  propose. 
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